docs: add cost-awareness implementation plan #24
No reviewers
Labels
No labels
code-review
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
sam/brainless!24
Loading…
Reference in a new issue
No description provided.
Delete branch "docs/cost-plan"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
Adds
cost_plan.md, the implementation plan for giving brainless agents a sense of token cost, in the same shape astime_plan.md(findings table -> tiers -> non-goals -> risks -> milestones -> decisions).Docs only; no code changes.
Why
Time awareness (PR #23, merged as
9b014ab) gave agents a clock. Cost is the second factor that should guide how much work to spend on a task: on a paid model every input and output token spends the operator's money, while a local model costs only time.Shape of the plan
brainless.json, per model:inputPerMillion,cachedInputPerMillion(optional, defaults toinputPerMillion= no cache discount),outputPerMillion,currency; or"free": truefor local models. No provider-level inheritance, and no cache-write rate (no write token count exists on any path brainless uses).<brainless-clock>block (preserving the prompt-prefix cache property from the time branch), a pureusageCostfunction, turn-log fields, a session ledger seeded once per process from persisted events, and compaction spend attributed through a summarizer decorator.promptTimeout.InvocationID.Correctness points found while verifying
PromptTokenCountincludes cached tokens, so cache reads must be subtracted, not added.candidates/output_tokens, soThoughtsTokenCountmust never be added on top of it.selectorModelusage wrapper, so its spend is currently invisible to any token accounting built there.Risks called out
Cost visibility invites corner-cutting (skipping tests, guessing at file contents). The plan ships the guardrail sentence with the numbers and recommends measuring outcomes before tuning any threshold.
Doc review of cost_plan.md (
635d6db). I independently re-verified the claims that are checkable against the repo, the pinned deps (adk/v2@v2.5.0, genai@v1.72.0, openai-go/v3@v3.71.1), and time_plan.md.Verified clean (no action): all harness-internal refs — turnclock.go:265
addUsage(prompt+cached only), fractions at :201-202; registry.go:280/:315/:329; harness.go:240 (static Instruction), :250-256 (clock wiring), :495-513 (turn log), :573-578 (run loop + expired-context branch), :1006-1028 (seedLastPromptEnded), :1226-1256 (compactionFor bypass), :1269; config.go:46/:67/:188. ADK refs — session.go:100-101 (Event embeds LLMResponse), :108 (InvocationID, also set at :242 in NewEvent); storage_session.go:220/:289-291/:381; compaction.go SummarizeResult.Usage; compactor.go:176/:199. Arithmetic rules — "PromptTokenCount includes cached" and "total = prompt+candidates+toolUse+thoughts" both match the genai comments; the output formula is consistent across both provider conventions and the "never add ThoughtsTokenCount" rule is right.Two real defects, both inside the "verified, not assumed" table the plan's credibility rests on:
The cache-write finding is wrong at the SDK level (lines 42, 131, 436, 498): openai-go v3.71.1 does expose
cache_write_tokens—CompletionUsagePromptTokensDetails.CacheWriteTokensat completion.go:241 (inside the cited range 237-243) andResponseUsageInputTokensDetails.CacheWriteTokensat responses/response.go:25678. What actually drops it is ADK'sconvertUsage(completions/response.go:149, responses/response.go:311 map only CachedTokens). The plan's conclusion (can't bill writes without ADK changes; overcount is the safe direction for OpenAI-style pricing) still holds, but "Verified absent" is over-strong and the barrier to write accounting is one bridge mapping, not a missing counter.The genai line numbers are from v1.71.0, not the pinned v1.72.0 (lines 38, 40, 41, 211): in v1.72.0 the usage struct is at types.go:3670-3700, PromptTokenCount at :3684, TotalTokenCount at :3698 — every citation is exactly 28 lines stale. Semantics are unchanged; the refs just need refreshing.
Minor: ADK bridge first-line refs off by 2 (completions :149 not :147, responses :311 not :309); the header "Branch: feat/cost-awareness" names a branch that doesn't exist yet (this PR is docs/cost-plan).
Verdict: strong plan, well-cross-referenced to the merged time work; fix the two table rows and it's mergeable. Details in the inline comments.
@ -0,0 +12,4 @@model response passes. Cost is a third counter in that same accumulator and athird line in that same block; it is not a new subsystem.Branch: `feat/cost-awareness`, based on `main` at `9b014ab`.Nit:
feat/cost-awarenessdoesn't exist yet - this PR is ondocs/cost-plan. time_plan.md's header named the branch it was committed on; here the plan is reviewed separately from the implementation. Consider "Planned implementation branch:feat/cost-awareness" or dropping the line until the code branch exists.@ -0,0 +35,4 @@| A per-round request hook already exists and already appends the clock block to the tail of the outgoing request | `internal/harness/turnclock.go` (`annotate`), wired at `internal/harness/harness.go:250-256` || Every model response in this repo passes one choke point, which already captures `UsageMetadata` | `selectorModel.GenerateContent`, `internal/harness/registry.go:280`, capture at `:315`, fold-in at `:329` || A per-turn accumulator for prompt/cached tokens already exists | `turnClock.addUsage`, `internal/harness/turnclock.go:265` || The genai usage struct carries prompt, cached, candidate, thought, tool-use and total counts | `google.golang.org/genai@v1.72.0/types.go:3642-3673` |Stale-version citations: these line numbers are from genai v1.71.0, not the pinned v1.72.0 that this row itself names and that go.mod pins. In v1.72.0 the struct is at types.go:3670-3700 (fields: CachedContentTokenCount :3675, CandidatesTokenCount :3678, PromptTokenCount :3684, ThoughtsTokenCount :3689, ToolUsePromptTokenCount :3692, TotalTokenCount :3698) - every reference in this table is exactly 28 lines short. Same drift in rows 40 and 41 (and line 211): the v1.72.0 equivalents are types.go:3695-3697 for the "total = sum including thoughts" comment and types.go:3682-3684 for "PromptTokenCount includes cached tokens". Semantics are unchanged between versions, so only the numbers need refreshing - but for a table selling "verified, not assumed" precision, a reader on the pinned version lands on GenerateContentResponsePromptFeedback instead of the usage struct.
@ -0,0 +39,4 @@| ADK's OpenAI bridge fills those from `prompt_tokens`/`completion_tokens`/`cached_tokens`/`reasoning_tokens` | `adk model/openaimodel/internal/completions/response.go:144-158`; Responses-API equivalent `.../responses/response.go:306-320` || On both OpenAI paths `candidate` means the raw `completion_tokens`/`output_tokens`, which **already include reasoning tokens** (the thought count is an overlapping subset, not an addition) | same two functions; genai-native semantics differ (`types.go:3668-3670` sums them) || `PromptTokenCount` **includes** cached tokens, so cache reads must be subtracted, not added | `genai types.go:3654-3656` || There is no cache-**write**/creation token counter anywhere on the paths brainless uses: genai exposes only `CachedContentTokenCount`, and the OpenAI bridge reads only `prompt_tokens_details.cached_tokens` / `input_tokens_details.cached_tokens` | `genai types.go:3642-3673`; `github.com/openai/openai-go/v3@v3.71.1/completion.go:237-243`; ADK mappings cited above |This finding is wrong at the SDK level, and the cited lines contain the counter it denies.
CompletionUsagePromptTokensDetails(completion.go:237-243) definesCacheWriteTokens int64 json:"cache_write_tokens"at completion.go:241 andCachedTokensat :243; the Responses-API equivalentResponseUsageInputTokensDetails.CacheWriteTokensis at responses/response.go:25678. The wire JSON and openai-go both carry the write count. What actually drops it is ADK's bridge:convertUsagemaps onlyPromptTokensDetails.CachedTokens->CachedContentTokenCount(completions/response.go:149) andInputTokensDetails.CachedTokens(responses/response.go:311);CacheWriteTokensis never read.So the correct statement is: genai has no write counter, and ADK's bridge discards the one the SDK already parses. That changes the plan's prerequisite text ("add write accounting only when a provider path reports a creation count"): the provider path already reports it - the missing piece is a two-line mapping in ADK's convertUsage (patch/fork or upstream fix). The conclusion (a
cacheWritePerMillionrate is dead config today, overcounting is the safe direction for OpenAI-style automatic caching) is unaffected. Please also soften the same over-strong claim at lines 131, 436 ("Verified absent"), and 498.@ -0,0 +217,4 @@once; if a future provider reports total greater than the sum (the genai-nativeconvention where thinking sits outside `candidates`), the remainder adds it.**Never add `ThoughtsTokenCount` on top of `candidates`** — on the two ADKpaths brainless uses it is already inside (`completions/response.go:147,156`,Nit: the first line ref of each pair is off by two in adk/v2@v2.5.0 - CachedContentTokenCount is mapped at completions/response.go:149 (not :147) and responses/response.go:311 (not :309). The ThoughtsTokenCount refs (:156, :318) are correct.
Resolved all four review findings on head
b8a9d1b(one commit: 32 insertions, 26 deletions incost_plan.md).1. Cache-write finding (comments 901, 903) — confirmed, I was wrong at the SDK level. openai-go v3.71.1 does parse the write count:
CompletionUsagePromptTokensDetails.CacheWriteTokens(cache_write_tokens) atcompletion.go:241, andResponseUsageInputTokensDetails.CacheWriteTokensatresponses/response.go:25678. What drops it is ADK'sconvertUsage, which maps onlyCachedTokens(completions/response.go:149,responses/response.go:311). Reworded at the findings table, tier 1a (both bullets), the cache-write non-goal, and decision 1. The conclusion is unchanged (acacheWritePerMillionrate is dead config today; overcounting is the safe direction for OpenAI-style pricing), but the prerequisite is corrected: it is a small mapping in ADK's bridge, upstream or forked, not a missing counter.2. genai version drift (comment 902) — confirmed. My line numbers were from v1.71.0. Refreshed to the pinned v1.72.0: struct
types.go:3670-3700,PromptTokenCount:3682-3684, the total-sum comment:3696-3698. All four references in the findings table plus the tier 1b rule are updated.3. ADK bridge refs (comment 903) — the cited pair was correct, but ambiguous.
completions/response.go:147,156andresponses/response.go:309,318point atCandidatesTokenCountandThoughtsTokenCount, not at the cached line;:149and:311areCachedContentTokenCount. So the "it is already inside" claim was correctly sourced. The ambiguity was fair, though, so each ref now names the field it holds and the sentence spells out that the candidate field is the raw completion/output count with the thought field a subset of it.4. Branch header (comment 904) — fixed. The document now says it is reviewed on
docs/cost-planand namesfeat/cost-awarenessonly as the planned implementation branch.Resolved all four review findings on head
b8a9d1b.1. Cache-write finding (comments 901, 903) — confirmed, I was wrong at the SDK level. openai-go v3.71.1 does parse the write count:
CompletionUsagePromptTokensDetails.CacheWriteTokens(cache_write_tokens) atcompletion.go:241, andResponseUsageInputTokensDetails.CacheWriteTokensatresponses/response.go:25678. What drops it is ADK'sconvertUsage, which maps onlyCachedTokens(completions/response.go:149,responses/response.go:311). Reworded at the findings table, tier 1a (both bullets), the cache-write non-goal, and decision 1. Conclusion unchanged (acacheWritePerMillionrate is dead config today; overcounting is the safe direction for OpenAI-style pricing); the prerequisite is corrected to a small ADK bridge mapping, upstream or forked.2. genai version drift (comment 902) — confirmed. My numbers were from v1.71.0. Refreshed to the pinned v1.72.0: struct
types.go:3670-3700,PromptTokenCount:3682-3684, the total-sum comment:3696-3698. All four findings-table refs plus the tier 1b rule are updated.3. ADK bridge refs (comment 903) — the pair was correct, but ambiguous.
completions/response.go:147,156andresponses/response.go:309,318point atCandidatesTokenCountandThoughtsTokenCount, not at the cached line;:149and:311areCachedContentTokenCount. So the "it is already inside" claim was correctly sourced. The ambiguity was fair, so each ref now names the field it holds and the sentence spells out that the candidate field is the raw completion/output count with the thought field a subset of it.4. Branch header (comment 904) — fixed. The plan now says it is reviewed on
docs/cost-planand namesfeat/cost-awarenessonly as the planned implementation branch.