feat: close out cost docs, cache-write accounting, context steering, fork, per-session cwd #26

Merged
sam merged 2 commits from feat/cost-docs-write-context-fork-cwd into main 2026-10-05 19:07:14 +00:00
Collaborator

Five small items closing the cost-awareness and session-lifecycle loops. Build, vet, full test suite, golangci-lint (0 issues), and the pure-Go racedetector are all green on the branch head.

1. Document the cost feature in the README

The pricing/budgets/history blocks shipped with zero user docs. Added a "Cost awareness" section covering prices (per-model, per-million, currency label, free-by-absence), the model-facing clock cost line, budgets (opt-in, priced models only), measured history, and the new contextWindow. Also refreshed the stale "Next steps" (session capabilities have been advertised since #25; fork and per-session cwd are done here) and the "what's consciously missing" bullet, and updated brainless.json.example + the Gemma wiring section.

2. Cache-write token accounting (cost_plan.md's explicit "add when")

ADK's convertUsage parses cache_write_tokens and discards it before genai sees it, and genai has no field for it. Instead of forking the whole ADK module for two lines, a middleware on each registry client (openai-go option.WithMiddleware) reads the count from the raw response body into a per-entry box before ADK decodes it; the selector (and the compaction costingLLM) folds it into the round. usageCost gains the third term: writes are uncached input billed at cacheWritePerMillion, which defaults to the fresh input rate (OpenAI's convention, so the default is a no-op). The operator log gains cache_write_tokens. Persisted events still lose the count, so a restart's seeded session total prices writes at the input rate — documented in cost_plan.md, which is updated to mark the non-goal resolved.

3. Context-size steering for free local models

contextWindow on a model (tokens) makes a free model's clock block show context: 31k / 128k tokens instead of a money line — the natural home the plan parked. The local Gemma example in the README and brainless.json.example now declare contextWindow: 4096.

4. session/fork

UnstableForkSession (the ACP experimental method): the fork gets a fresh id, a verbatim copy of the source conversation's events (ids, timestamps, state deltas intact), its own persisted cwd, becomes the active session, and connects the requested MCP servers. Advertised via sessionCapabilities.fork in initialize.

5. Per-session cwd

NewSession now creates the session row up front with the working directory in the session state (brainless.cwd key), so session/list reports each session's own cwd — Paseo feeds it back into session/load and its import UI — across restarts. session/load/resume set the dir from the request, and a restart-resumed prompt restores the session's own cwd for the file tools. Sessions created but never prompted now appear in the list (they exist in the store; that is what session/list means); sessions recorded before this feature fall back to the most recent dir.

Tests added

  • usageCost cache-write cases (billed at the write rate, default-to-input no-op, clamped out of a fully cached prompt, free ignores writes)
  • clock context line (window alone before usage, prompt/window after, absent for priced or window-less models)
  • write accumulator + cacheWritePerMillion/write-rate-default config validation
  • middleware capture (readCacheWriteTokens shapes + end-to-end through a real registry client)
  • e2e: per-session cwd survives a restart; fork copies history, reports its own cwd, leaves the source untouched; a money budget crossed only by the write premium cancels the turn (proves the whole middleware → usageCost → budget chain); initialize advertises fork
Five small items closing the cost-awareness and session-lifecycle loops. Build, vet, full test suite, golangci-lint (0 issues), and the pure-Go racedetector are all green on the branch head. ## 1. Document the cost feature in the README The `pricing`/`budgets`/`history` blocks shipped with zero user docs. Added a "Cost awareness" section covering prices (per-model, per-million, currency label, free-by-absence), the model-facing clock cost line, budgets (opt-in, priced models only), measured history, and the new `contextWindow`. Also refreshed the stale "Next steps" (session capabilities have been advertised since #25; fork and per-session cwd are done here) and the "what's consciously missing" bullet, and updated `brainless.json.example` + the Gemma wiring section. ## 2. Cache-write token accounting (cost_plan.md's explicit "add when") ADK's `convertUsage` parses `cache_write_tokens` and discards it before `genai` sees it, and `genai` has no field for it. Instead of forking the whole ADK module for two lines, a middleware on each registry client (openai-go `option.WithMiddleware`) reads the count from the raw response body into a per-entry box before ADK decodes it; the selector (and the compaction `costingLLM`) folds it into the round. `usageCost` gains the third term: writes are uncached input billed at `cacheWritePerMillion`, which defaults to the fresh input rate (OpenAI's convention, so the default is a no-op). The operator log gains `cache_write_tokens`. Persisted events still lose the count, so a restart's seeded session total prices writes at the input rate — documented in `cost_plan.md`, which is updated to mark the non-goal resolved. ## 3. Context-size steering for free local models `contextWindow` on a model (tokens) makes a free model's clock block show `context: 31k / 128k tokens` instead of a money line — the natural home the plan parked. The local Gemma example in the README and `brainless.json.example` now declare `contextWindow: 4096`. ## 4. session/fork `UnstableForkSession` (the ACP experimental method): the fork gets a fresh id, a verbatim copy of the source conversation's events (ids, timestamps, state deltas intact), its own persisted cwd, becomes the active session, and connects the requested MCP servers. Advertised via `sessionCapabilities.fork` in `initialize`. ## 5. Per-session cwd `NewSession` now creates the session row up front with the working directory in the session state (`brainless.cwd` key), so `session/list` reports each session's own cwd — Paseo feeds it back into `session/load` and its import UI — across restarts. `session/load`/`resume` set the dir from the request, and a restart-resumed prompt restores the session's own cwd for the file tools. Sessions created but never prompted now appear in the list (they exist in the store; that is what `session/list` means); sessions recorded before this feature fall back to the most recent dir. ## Tests added - `usageCost` cache-write cases (billed at the write rate, default-to-input no-op, clamped out of a fully cached prompt, free ignores writes) - clock context line (window alone before usage, prompt/window after, absent for priced or window-less models) - write accumulator + `cacheWritePerMillion`/write-rate-default config validation - middleware capture (`readCacheWriteTokens` shapes + end-to-end through a real registry client) - e2e: per-session cwd survives a restart; fork copies history, reports its own cwd, leaves the source untouched; a money budget crossed only by the write premium cancels the turn (proves the whole middleware → usageCost → budget chain); initialize advertises fork
feat: close out cost docs, cache-write accounting, context steering, fork, per-session cwd
All checks were successful
code-review / Agent review (pull_request) Successful in 9s
e912043470
Five small items that close the cost-awareness and session-lifecycle loops:

1. Document the cost feature in the README: pricing, budgets, history, and
   contextWindow blocks, the model-facing clock line, and refreshed next-steps
   (session capabilities, fork, and per-session cwd are all done now).

2. Cache-write token accounting (cost_plan.md's parked "add when"): ADK's
   convertUsage parses cache_write_tokens then discards it before genai sees
   it, so a middleware on each registry client reads the count from the raw
   response into a per-entry box, and usageCost bills it as a third term at
   cacheWritePerMillion (defaulting to the fresh input rate). The live ledger
   is exact on providers that bill writes at a premium; the seeded restart
   total still prices writes at the input rate because persisted events lose
   the count.

3. Context-size steering for free local models: a per-model contextWindow
   makes the clock block show "context: 31k / 128k tokens" instead of a
   money line, so a free model is steered by prefill time.

4. session/fork: UnstableForkSession branches a session into a fresh id with
   a verbatim copy of its events, advertises the fork capability, and reports
   its own cwd.

5. Per-session cwd: NewSession creates the session row up front with the
   working directory in session state, so session/list reports each session's
   own cwd (Paseo feeds it back into session/load and its import UI), load
   and resume set the dir, and a restart-resumed prompt restores it.
pi-reviewer left a comment

Reviewed at head e912043 (feat/cost-docs-write-context-fork-cwd) with the full toolchain, not from memory. Verified in this session: go build ./..., go vet ./..., go test ./... (all packages), racedetector test ./... (pure-Go race detector, no data races), golangci-lint run (0 issues), govulncheck (0 reachable vulnerabilities; no go.mod/go.sum changes and no replace directives in the PR).

Substantive checks that came back clean:

  • Middleware/box correctness — the cacheWriteMiddleware signature matches openai-go v3.71.1 option.WithMiddleware(func(*http.Request, option.MiddlewareNext) (*http.Response, error)) (option/requestoption.go:102). The middleware runs inside the retry loop, so the box ends up holding the last attempt's count — the attempt whose response is decoded — and take() drains it before the next call, so stale counts cannot be billed. Calls are serialized by turnMu (Prompt holds it for the whole turn; startMCP takes it internally; compaction runs synchronously inside the runner), so the shared box between the session client and the summarizer is safe.
  • Fork conversion helpers — UnstableMcpServer↔McpServer and the config-option variants serialize to identical JSON (verified against acp-go-sdk v0.13.5 types_gen.go: same type discriminator, same field names), so the marshal/unmarshal round trips are faithful. The SDK validates Cwd server-side before dispatch, and SessionForkCapabilities exists.
  • Money math — usageCost write handling is correct for the OpenAI convention (write tokens inside the uncached input; default write rate = input rate makes the subtraction a no-op), nil/free pricing short-circuits before CacheWriteRate(), and the arithmetic is pinned by the new table cases including the fully-cached clamp.
  • Concurrency — tools.Dir is RWMutex-protected; writeTokensBox is atomic.Int64; no goroutines added. The e2e suite (per-session cwd across restart, fork copy/replay, write-premium budget cancellation) runs green, including under the race detector.

Three non-blocking suggestions are inline (🟡). No blocking findings — approving.

Reviewed at head e912043 (feat/cost-docs-write-context-fork-cwd) with the full toolchain, not from memory. Verified in this session: `go build ./...`, `go vet ./...`, `go test ./...` (all packages), `racedetector test ./...` (pure-Go race detector, no data races), `golangci-lint run` (0 issues), `govulncheck` (0 reachable vulnerabilities; no go.mod/go.sum changes and no `replace` directives in the PR). Substantive checks that came back clean: - **Middleware/box correctness** — the `cacheWriteMiddleware` signature matches openai-go v3.71.1 `option.WithMiddleware(func(*http.Request, option.MiddlewareNext) (*http.Response, error))` (option/requestoption.go:102). The middleware runs inside the retry loop, so the box ends up holding the last attempt's count — the attempt whose response is decoded — and `take()` drains it before the next call, so stale counts cannot be billed. Calls are serialized by `turnMu` (Prompt holds it for the whole turn; `startMCP` takes it internally; compaction runs synchronously inside the runner), so the shared box between the session client and the summarizer is safe. - **Fork conversion helpers** — `UnstableMcpServer`↔`McpServer` and the config-option variants serialize to identical JSON (verified against acp-go-sdk v0.13.5 types_gen.go: same `type` discriminator, same field names), so the marshal/unmarshal round trips are faithful. The SDK validates `Cwd` server-side before dispatch, and `SessionForkCapabilities` exists. - **Money math** — `usageCost` write handling is correct for the OpenAI convention (write tokens inside the uncached input; default write rate = input rate makes the subtraction a no-op), nil/free pricing short-circuits before `CacheWriteRate()`, and the arithmetic is pinned by the new table cases including the fully-cached clamp. - **Concurrency** — `tools.Dir` is RWMutex-protected; `writeTokensBox` is `atomic.Int64`; no goroutines added. The e2e suite (per-session cwd across restart, fork copy/replay, write-premium budget cancellation) runs green, including under the race detector. Three non-blocking suggestions are inline (🟡). No blocking findings — approving.
@ -636,1 +688,4 @@
// working directory, so relative file paths resolve where the session
// was created.
if h.clock != nil {
h.ensureSessionCwd(ctx, sid)
Collaborator

🟡 SUGGESTION — ensureSessionCwd runs on every prompt, not only for sessions this process never opened. Its doc comment says it restores the cwd "when a prompt arrives on a session this process never opened with a lifecycle call", but the call at harness.go:691 is unconditional under h.clock != nil: even after NewSession/LoadSession/ResumeSession set h.dir from the client's request, the next Prompt silently overwrites it with the persisted creation cwd. A client that loads a session with a different cwd (e.g. the workspace moved) is reverted at prompt time, and every prompt pays an extra store.Get. Concrete fix: remember the session ids this process already opened via a lifecycle call and skip the restore for them (or only restore when the process is resuming a session it never opened).

🟡 SUGGESTION — `ensureSessionCwd` runs on every prompt, not only for sessions this process never opened. Its doc comment says it restores the cwd "when a prompt arrives on a session this process never opened with a lifecycle call", but the call at harness.go:691 is unconditional under `h.clock != nil`: even after NewSession/LoadSession/ResumeSession set `h.dir` from the client's request, the next Prompt silently overwrites it with the persisted creation cwd. A client that loads a session with a different cwd (e.g. the workspace moved) is reverted at prompt time, and every prompt pays an extra `store.Get`. Concrete fix: remember the session ids this process already opened via a lifecycle call and skip the restore for them (or only restore when the process is resuming a session it never opened).
@ -272,0 +341,4 @@
// usage.prompt_tokens_details.
//
//nolint:ireturn // openai-go's option type is an interface by design
func cacheWriteMiddleware(box *writeTokensBox) option.RequestOption {
Collaborator

🟡 SUGGESTION — the capture only works for non-streaming responses, and the constraint is undocumented. io.ReadAll(res.Body) on an SSE stream blocks until the stream ends (defeating incremental delivery), and readCacheWriteTokens cannot parse a multi-object SSE body (json.Unmarshal of the whole blob fails → 0), so cache-write tokens are silently unbilled on a streamed call. Not reachable today — the runner pins agent.StreamingModeNone (harness.go:711) and ADK only streams when rc.StreamingMode == agent.StreamingModeSSE (adk/v2 base_flow.go:975), so useStream is always false — but the middleware is installed unconditionally on every registry client. Consider a one-line doc note (or a guard) so a future streaming path does not quietly lose the count.

🟡 SUGGESTION — the capture only works for non-streaming responses, and the constraint is undocumented. `io.ReadAll(res.Body)` on an SSE stream blocks until the stream ends (defeating incremental delivery), and `readCacheWriteTokens` cannot parse a multi-object SSE body (`json.Unmarshal` of the whole blob fails → 0), so cache-write tokens are silently unbilled on a streamed call. Not reachable today — the runner pins `agent.StreamingModeNone` (harness.go:711) and ADK only streams when `rc.StreamingMode == agent.StreamingModeSSE` (adk/v2 base_flow.go:975), so `useStream` is always false — but the middleware is installed unconditionally on every registry client. Consider a one-line doc note (or a guard) so a future streaming path does not quietly lose the count.
@ -506,3 +545,1 @@
// - money = (uncachedInput*input + cachedInput*cachedInput + output*output)
// / 1e6, where an omitted cachedInputPerMillion defaults to the fresh
// input rate (no cache discount assumed; the conservative direction).
// - money = (uncachedInput*input + cachedInput*cachedInput + write*writeRate
Collaborator

🟡 SUGGESTION — the wrapped formula reads as subtraction. The continuation line starts with the list marker -, so money = (… + write*writeRate / - output*output) / 1e6 visually reads as write*writeRate − output*output, and the + before the output term is missing (verified in the head file, lines 545-546). Fix: put the whole formula on one line, e.g. money = (uncachedInput*input + cachedInput*cachedInput + write*writeRate + output*output) / 1e6, or indent the continuation so the bullet marker is not inside the expression.

🟡 SUGGESTION — the wrapped formula reads as subtraction. The continuation line starts with the list marker `-`, so `money = (… + write*writeRate` / `- output*output) / 1e6` visually reads as `write*writeRate − output*output`, and the `+` before the output term is missing (verified in the head file, lines 545-546). Fix: put the whole formula on one line, e.g. `money = (uncachedInput*input + cachedInput*cachedInput + write*writeRate + output*output) / 1e6`, or indent the continuation so the bullet marker is not inside the expression.
Three non-blocking findings from the PR #26 review, all resolved:

1. turnclock.go: the usageCost doc formula wrapped onto a continuation line
   that rendered as a nested list item, reading as "write*writeRate - output"
   with the + lost. The continuation is now indented with its + intact.

2. harness.go: ensureSessionCwd ran on every prompt, silently reverting a
   client-set cwd (a load with a moved workspace) and paying a store.Get per
   prompt. Lifecycle calls (new/load/resume/fork) now mark the session id as
   opened, and the restore is gated on that, so only sessions this process
   never opened get their persisted cwd back. New TestRestoreSessionCwd pins
   both halves.

3. registry.go: the cache-write middleware's non-streaming constraint is now
   documented on the middleware itself, so a future streaming path cannot
   silently lose the count.
Author
Collaborator

All three 🟡 findings from review 51 are addressed at head 4db61b8 (fix: address review findings (doc formula, cwd restore gate, streaming note)):

  1. turnclock.go formula — the doc continuation now reads + output*output on an indented line, so it no longer renders as a nested list item (write*writeRate − output).
  2. ensureSessionCwd on every prompt — lifecycle calls (new/load/resume/fork) now mark the session id in a process-local opened set (guarded by h.mu), and restoreSessionCwd only restores for sessions never opened: a client that loads a session with a different cwd (moved workspace) keeps it, and opened sessions no longer pay a store.Get per prompt. New TestRestoreSessionCwd pins both halves.
  3. streaming constraint — the middleware's doc comment now states the capture is non-streaming only (whole-body buffer; multi-object SSE body would not parse) and that it is unreachable while the runner pins agent.StreamingModeNone.

Verified at the new head: go build, go vet, go test ./..., golangci-lint run (0 issues), racedetector test ./internal/harness/ (no data races).

All three 🟡 findings from review 51 are addressed at head `4db61b8` (`fix: address review findings (doc formula, cwd restore gate, streaming note)`): 1. **turnclock.go formula** — the doc continuation now reads `+ output*output` on an indented line, so it no longer renders as a nested list item (`write*writeRate − output`). 2. **ensureSessionCwd on every prompt** — lifecycle calls (new/load/resume/fork) now mark the session id in a process-local `opened` set (guarded by `h.mu`), and `restoreSessionCwd` only restores for sessions never opened: a client that loads a session with a different cwd (moved workspace) keeps it, and opened sessions no longer pay a `store.Get` per prompt. New `TestRestoreSessionCwd` pins both halves. 3. **streaming constraint** — the middleware's doc comment now states the capture is non-streaming only (whole-body buffer; multi-object SSE body would not parse) and that it is unreachable while the runner pins `agent.StreamingModeNone`. Verified at the new head: `go build`, `go vet`, `go test ./...`, `golangci-lint run` (0 issues), `racedetector test ./internal/harness/` (no data races).
sam merged commit 5943d68a68 into main 2026-10-05 19:07:14 +00:00
sam deleted branch feat/cost-docs-write-context-fork-cwd 2026-10-05 19:07:14 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!26
No description provided.