feat: add per-model context compaction #21

Merged
sam merged 4 commits from feat/context-compaction into main 2026-10-04 07:58:03 +00:00
Collaborator

Summary

Adds session context compaction to the harness: once a session's prompt grows past a token budget, ADK summarizes the older turns and later prompts carry the summary plus the most recent events. ADK v2.4.0 (already a dependency) ships the whole mechanism — tail retention, an LLM summarizer, and SQLite persistence of summary events — so brainless only wires the runner option and resolves the config.

Compaction is on by default (ADK tail retention: one rolling summary + the 10 most recent events). An optional compaction block in brainless.json tunes it:

{
  "compaction": {
    "defaultThreshold": 20000,
    "compactionThreshold": {
      "local/gemma4:e2b": 3000
    },
    "summarizerModel": "melious/deepseek-v4-flash-0731"
  }
}
  • defaultThreshold — prompt token count that triggers summarization, for models without their own entry (built-in default 20000).
  • compactionThreshold — per-model overrides keyed by provider/model, because context windows differ: a small-context model (e.g. local Gemma 4 E2B ~4096 tokens) compacts well below its window, a 128k cloud model stays on the default.
  • summarizerModel — the configured model that writes summaries; defaults to the session's own model, but a model too small to summarize its own history can point this at a larger-context model.
  • "disabled": true turns compaction off. (Schema note: the toggle is disabled, not enabled, so a plain bool keeps "absent means on" without a pointer-to-bool.)

Because the threshold is per model and ADK reads it from a fixed runner config, a model switch rebuilds the runner with the new model's threshold. The runner is stateless (sessions live in the store) and the root agent is shared, so the rebuild is cheap; it happens under the existing turn lock in Prompt, where the model selection already takes effect. The prompt-before-initialize check moved under the same lock since the runner is now mutable.

Invalid settings fail at boot: negative thresholds, non-positive per-model thresholds, and threshold/summarizerModel keys that name no configured model. A failed summarization is logged and the turn continues; it never fails the user's answer.

Testing

  • New unit tests: threshold resolution (default / per-model override / fallback), Disabled flag, summarizer-model selection, validation errors, JSON parsing of the block.
  • go build, go vet, gofmt, full go test ./..., golangci-lint (0 issues) all pass; the TestACP* e2e tests exercise the real Prompt path including the runner swap after a model switch.

Reviewer notes

Review comments from the first pass are resolved (see fix: resolve review comments on context compaction): no pointer-to-bool (disabled flag + omitzero), shortened comments, the runner-swap race fixed by moving the nil check under turnMu, ok from reg.Default() handled, and the (nil, nil) + nilnil nolint eliminated via a compactionEnabled() gate. The *compaction.Config pointer remains because ADK's runner.Config.Compaction is a pointer and nil is its "disabled" value.

## Summary Adds session context compaction to the harness: once a session's prompt grows past a token budget, ADK summarizes the older turns and later prompts carry the summary plus the most recent events. ADK v2.4.0 (already a dependency) ships the whole mechanism — tail retention, an LLM summarizer, and SQLite persistence of summary events — so brainless only wires the runner option and resolves the config. Compaction is **on by default** (ADK tail retention: one rolling summary + the 10 most recent events). An optional `compaction` block in `brainless.json` tunes it: ```json { "compaction": { "defaultThreshold": 20000, "compactionThreshold": { "local/gemma4:e2b": 3000 }, "summarizerModel": "melious/deepseek-v4-flash-0731" } } ``` - `defaultThreshold` — prompt token count that triggers summarization, for models without their own entry (built-in default 20000). - `compactionThreshold` — per-model overrides keyed by `provider/model`, because context windows differ: a small-context model (e.g. local Gemma 4 E2B ~4096 tokens) compacts well below its window, a 128k cloud model stays on the default. - `summarizerModel` — the configured model that writes summaries; defaults to the session's own model, but a model too small to summarize its own history can point this at a larger-context model. - `"disabled": true` turns compaction off. (Schema note: the toggle is `disabled`, not `enabled`, so a plain bool keeps "absent means on" without a pointer-to-bool.) Because the threshold is per model and ADK reads it from a fixed runner config, a model switch rebuilds the runner with the new model's threshold. The runner is stateless (sessions live in the store) and the root agent is shared, so the rebuild is cheap; it happens under the existing turn lock in `Prompt`, where the model selection already takes effect. The `prompt-before-initialize` check moved under the same lock since the runner is now mutable. Invalid settings fail at boot: negative thresholds, non-positive per-model thresholds, and threshold/summarizerModel keys that name no configured model. A failed summarization is logged and the turn continues; it never fails the user's answer. ## Testing - New unit tests: threshold resolution (default / per-model override / fallback), `Disabled` flag, summarizer-model selection, validation errors, JSON parsing of the block. - `go build`, `go vet`, `gofmt`, full `go test ./...`, `golangci-lint` (0 issues) all pass; the `TestACP*` e2e tests exercise the real `Prompt` path including the runner swap after a model switch. ## Reviewer notes Review comments from the first pass are resolved (see `fix: resolve review comments on context compaction`): no pointer-to-bool (`disabled` flag + `omitzero`), shortened comments, the runner-swap race fixed by moving the nil check under `turnMu`, `ok` from `reg.Default()` handled, and the `(nil, nil)` + `nilnil` nolint eliminated via a `compactionEnabled()` gate. The `*compaction.Config` pointer remains because ADK's `runner.Config.Compaction` is a pointer and nil is its "disabled" value.
Compaction keeps a session's prompt small as it grows: once the prompt
passes a token budget, ADK summarizes the older turns and later prompts
carry the summary plus the most recent events. ADK v2.4.0 ships the whole
mechanism (tail retention, LLM summarizer, SQLite persistence of summary
events); brainless only wires the runner option and resolves the config.

Compaction is on by default; an optional brainless.json block tunes it:

- defaultThreshold: prompt token count that triggers summarization, for
  models without their own entry (built-in default 20000).
- compactionThreshold: per-model overrides keyed by provider/model, so a
  small-context model (e.g. local Gemma 4 E2B ~4096 tokens) compacts well
  below its window while a 128k cloud model stays on the default.
- summarizerModel: the configured model that writes summaries; defaults to
  the session's own model, but a model too small to summarize its own
  history can point this at a larger-context model.
- enabled: false turns compaction off.

Because the threshold is per model and ADK reads it from a fixed runner
config, a model switch rebuilds the runner with the new model's threshold.
The runner is stateless (sessions live in the store) and the root agent is
shared, so the rebuild is cheap; it happens under the existing turn lock
in Prompt, where the model selection already takes effect.

Invalid settings fail at boot: negative thresholds, non-positive per-model
thresholds, and threshold/summarizerModel keys that name no configured
model. A failed summarization is logged and the turn continues; it never
fails the user's answer.
fix: resolve review comments on context compaction
All checks were successful
code-review / Agent review (pull_request) Successful in 9s
8a6d6c2b84
- Replace the *bool Enabled field with a plain bool Disabled flag and
  omitzero tags, so 'absent means on' survives without pointer-to-bool.
- Shorten the new comments.
- Take turnMu before the prompt-before-initialize check: the runner is
  now mutable (swapped on model change), so every runner access must be
  under the lock.
- Handle the ok from reg.Default() instead of discarding it.
- Split compactionEnabled() out of compactionFor() so the helper never
  returns (nil, nil) and the nilnil nolint is gone; the *compaction.Config
  pointer stays because ADK's runner.Config.Compaction is a pointer and
  nil is its 'disabled' value.
pi-reviewer left a comment

AI-driven code review — context compaction (feat/context-compaction)

Reviewed with the samber/cc-skills-golang skill set (code-style/naming/docs → error-handling/safety/concurrency → security/dependency-management → testing/performance/observability/modernize), every claim verified in this session against the PR head 8a6d6c2 (detached worktree, no local-checkout assumptions).

Verification gate (commands actually run)

  • go build ./... → exit 0
  • go vet ./... → exit 0
  • go test ./... -count=1 → all packages ok
  • racedetector test ./internal/harness/ -count=1 → ok, no data races
  • golangci-lint run ./internal/config/... ./internal/harness/... → 0 issues (matches the PR claim)
  • gofmt -l internal/ → clean
  • go doc google.golang.org/adk/v2/session/compaction (+ Config, LLMSummarizerConfig, NewLLMSummarizer, ErrCompaction) and go doc .../runner.Config → every ADK symbol the diff relies on exists with the exact shape used, in the version the PR actually builds against (go list -m google.golang.org/adk/v2 → v2.4.0; the diff changes no go.mod/go.sum lines, so no new dependency, no version bump, no replace directive)

Findings

  • 🔴 BLOCKING: none.
  • 🟠 IMPORTANT: none.
  • 🟡 SUGGESTION: 1 (inline comment on internal/harness/compaction_test.go — dangling //go:fix inline directive).

Notes (verified, no action needed)

  • The runner-rebuild-on-model-switch is race-free: h.runner/h.runnerModel are touched only under turnMu in Prompt and once in New (grep of all h.runner sites); the nil check moved under the same lock; race detector is clean.
  • The errors.Is(err, compaction.ErrCompaction) + log-and-continue pattern is exactly ADK's documented usage (the go doc output for ErrCompaction shows this loop verbatim); the failure never fails the user's answer.
  • compactionFor cannot produce an invalid compaction.Config at runtime: TokenThreshold is always > 0 (0 → built-in default 20000, negatives and non-positive per-model values rejected at boot by validateCompaction), and EventRetentionSize is always 10 — ADK's Config.Validate rejects a config enabling neither strategy, which is unreachable here.
  • TestACPRoundTripEndToEnd genuinely exercises the swap: it switches model p/m1 → p/m2 and then prompts through the real Prompt path.
  • The //nolint:ireturn on Registry.LLM carries an explanation on the same line (satisfies nolintlint), consistent with the existing activate precedent.
  • Compaction defaults-on with disabled bool: omitzero (Go 1.24+) on the bool/int/string/map fields is valid for the module's go 1.27.1 toolchain.

Solid PR: correct concurrency, faithful use of the ADK mechanism, strong test coverage, docs match the code. Approving — the single suggestion is optional cleanup.

## AI-driven code review — context compaction (feat/context-compaction) Reviewed with the samber/cc-skills-golang skill set (code-style/naming/docs → error-handling/safety/concurrency → security/dependency-management → testing/performance/observability/modernize), every claim verified in this session against the PR head `8a6d6c2` (detached worktree, no local-checkout assumptions). ### Verification gate (commands actually run) - `go build ./...` → exit 0 - `go vet ./...` → exit 0 - `go test ./... -count=1` → all packages ok - `racedetector test ./internal/harness/ -count=1` → ok, no data races - `golangci-lint run ./internal/config/... ./internal/harness/...` → 0 issues (matches the PR claim) - `gofmt -l internal/` → clean - `go doc google.golang.org/adk/v2/session/compaction` (+ `Config`, `LLMSummarizerConfig`, `NewLLMSummarizer`, `ErrCompaction`) and `go doc .../runner.Config` → every ADK symbol the diff relies on exists with the exact shape used, in the version the PR actually builds against (`go list -m google.golang.org/adk/v2` → v2.4.0; the diff changes no go.mod/go.sum lines, so no new dependency, no version bump, no replace directive) ### Findings - 🔴 BLOCKING: none. - 🟠 IMPORTANT: none. - 🟡 SUGGESTION: 1 (inline comment on `internal/harness/compaction_test.go` — dangling `//go:fix inline` directive). ### Notes (verified, no action needed) - The runner-rebuild-on-model-switch is race-free: `h.runner`/`h.runnerModel` are touched only under `turnMu` in `Prompt` and once in `New` (grep of all `h.runner` sites); the nil check moved under the same lock; race detector is clean. - The `errors.Is(err, compaction.ErrCompaction)` + log-and-`continue` pattern is exactly ADK's documented usage (the `go doc` output for `ErrCompaction` shows this loop verbatim); the failure never fails the user's answer. - `compactionFor` cannot produce an invalid `compaction.Config` at runtime: `TokenThreshold` is always > 0 (0 → built-in default 20000, negatives and non-positive per-model values rejected at boot by `validateCompaction`), and `EventRetentionSize` is always 10 — ADK's `Config.Validate` rejects a config enabling neither strategy, which is unreachable here. - `TestACPRoundTripEndToEnd` genuinely exercises the swap: it switches model `p/m1` → `p/m2` and then prompts through the real `Prompt` path. - The `//nolint:ireturn` on `Registry.LLM` carries an explanation on the same line (satisfies `nolintlint`), consistent with the existing `activate` precedent. - Compaction defaults-on with `disabled` bool: `omitzero` (Go 1.24+) on the bool/int/string/map fields is valid for the module's `go 1.27.1` toolchain. Solid PR: correct concurrency, faithful use of the ADK mechanism, strong test coverage, docs match the code. Approving — the single suggestion is optional cleanup.
@ -0,0 +215,4 @@
}
}
//go:fix inline
Collaborator

🟡 SUGGESTION — dangling //go:fix inline directive at EOF, remove the line.

go:fix is a real directive in the Go 1.27 toolchain (it is consumed by the x/tools inline analysis pass, proposal #32816), but it only takes effect when placed in a declaration's doc comment — the gofixdirective.Find pass scans FuncDecl/GenDecl doc comments, and this line follows the last } of TestValidateCompaction at file scope, so it annotates nothing.

Verified in this session: go build ./..., go vet ./..., and go test ./internal/harness/ all pass with it present, so it is harmless — but it reads as an accidental paste. Delete the line.

🟡 **SUGGESTION** — dangling `//go:fix inline` directive at EOF, remove the line. `go:fix` is a real directive in the Go 1.27 toolchain (it is consumed by the x/tools `inline` analysis pass, proposal #32816), but it only takes effect when placed in a declaration's doc comment — the `gofixdirective.Find` pass scans `FuncDecl`/`GenDecl` doc comments, and this line follows the last `}` of `TestValidateCompaction` at file scope, so it annotates nothing. Verified in this session: `go build ./...`, `go vet ./...`, and `go test ./internal/harness/` all pass with it present, so it is harmless — but it reads as an accidental paste. Delete the line.
Author
Collaborator

Resolved: the dangling //go:fix inline directive at the end of internal/harness/compaction_test.go is removed in 36c0edf (the file now ends at the closing } of TestValidateCompaction). It was an accidental artifact of the Go 1.27 auto-fixer, as the reviewer suspected. Verified: go build, go test ./internal/harness/, golangci-lint (0 issues), gofmt clean.

Resolved: the dangling `//go:fix inline` directive at the end of `internal/harness/compaction_test.go` is removed in `36c0edf` (the file now ends at the closing `}` of `TestValidateCompaction`). It was an accidental artifact of the Go 1.27 auto-fixer, as the reviewer suspected. Verified: `go build`, `go test ./internal/harness/`, `golangci-lint` (0 issues), `gofmt` clean.
sam merged commit e697f9024c into main 2026-10-04 07:58:03 +00:00
sam deleted branch feat/context-compaction 2026-10-04 07:58:03 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!21
No description provided.