feat: give agents a concept of time (clock, tool costs, timeout, preemption) #23
No reviewers
Labels
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
sam/brainless!23
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/time-awareness"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
Gives agents in the brainless harness a working sense of the passage of time, implemented across the four tiers of
time_plan.mdplus the Milestone 0 cancellation investigation.Tier 1 — live clock and usage telemetry. Every model request carries a trailing system-role
<brainless-clock>block: UTC now, elapsed time in the current task, round number, time since the user's last message, and a per-turnprompt completelog with rounds, prompt/cached tokens, cache-hit ratio, model time, and turn time. Cache-safe: the block is appended at the tail only, and a prefix-stability test pins that earlier content is never modified.Tier 2 — tool durations. Each tool call is timed from its FunctionCall to FunctionResponse event; the clock block reports
tools in this task: X total; slowest: NAME Y, the finished tool's result is tagged[took 4m2s]in the outgoing request, and the turn summary includes tool time and slowest call.Tier 3 — per-prompt timeout.
"timeouts": { "promptTimeout": "10m" }in brainless.json (Go duration string, boot-validated, off unless configured). The clock block gains a countdown at 50% of the budget and a directive at 90%; at expiry the turn settles with the partial answer and stop reasoncancelled, and the in-flight model call is aborted.Tier 4 — mid-turn preemption. A prompt sent while another is running preempts it at the next safe boundary: immediately when no tool call is in flight, otherwise once the tool finishes. A tool call is never severed; immediate interruption remains
session/cancel.Milestone 0 — ADK cancellation is not a bug. The long-standing "ADK does not abort in-flight model calls" claim was an artifact of a test server that never read its request body (Go's http server only starts the disconnect-watching background read once the body hits EOF). Pinned by
TestADKCancelAbortsInflightModelCallplus two bisection tests (plain net/http and bare openai-go). Tiers 3 and 4 rely on this.Also on this branch: a pre-existing flake fix — in-memory memory stores now get a unique per-instance database name, so parallel
cmd/brainlesstests no longer race each other'sCREATE TABLE(fix(tools): isolate per-instance in-memory memory stores).Commits
a9318afdocs: add time-awareness implementation plan8f6a08bfeat(harness): add live clock and usage telemetry to prompt turns22fb438fix(tools): isolate per-instance in-memory memory stores886f279test(harness): prove session/cancel aborts in-flight model calls (M0)b79de12feat(harness): report tool durations to the model (Tier 2)8dc917bfeat(harness): add per-prompt timeout with escalating warnings (Tier 3)e35d749feat(harness): preempt a running prompt at the next safe boundary (Tier 4)Verification
go build,go vet, fullgo test ./...(repeated),racedetectoron the touched packages,golangci-lint— all clean. New tests cover the clock block, prefix stability, tool timing/tagging, timeout parsing and escalation, prompt-timeout settling, and mid-turn preemption.Review — feat/time-awareness (PR #23)
Reviewed the full diff (README, config, harness, registry, turnclock, memory, time_plan.md, and all tests) as
pi-reviewer. Everything in this review was verified in this session against the head commite35d749and the pinned ADK v2.5.0 / openai-go v3.71.1 module sources, not from memory.Verified clean:
go build ./...,go vet, fullgo test ./internal/harness/ ./internal/tools/ ./internal/config/(incl. the newTestACPMidTurnPromptPreempts,TestACPPromptTimeoutSettlesTurn,TestADKCancelAbortsInflightModelCall,cancel_http_test.gopair, and the turnclock suite),racedetector test ./internal/harness/(no DATA RACE warnings),golangci-lint run(0 issues),govulncheck ./...(exit 0). Nogo.mod/go.sumchanges vsmain. The ADKBeforeModelCallbackcontract ((nil, nil)→ continue to the model call) is confirmed inagent/llmagent/llmagent.go:373-375, and the merged parallel-tool-response event ininternal/llminternal/base_flow.go:1614makes the booleantoolInFlighttracking correct for parallel tool calls.Findings: 1 IMPORTANT, 1 SUGGESTION (no BLOCKING).
🟠 IMPORTANT —
internal/harness/registry.go:329: theselectorModel.GenerateContentwrapper accumulates the compaction summarizer's usage into the running turn's clock whenevercompaction.summarizerModelis unset (the default). Verified:compactionForleavesout.Summarizernil in that case (internal/harness/harness.go:1200-1235), ADK's runner then installsNewLLMSummarizerover the root agent's model — thisselectorModel— (runner/runner.go:172-201), and the summarizer callss.model.GenerateContent(...)directly (llm_summarizer.go:232), landing in this wrapper. When compaction fires (default threshold 20k tokens — routine for long sessions),prompt_tokens/cached_tokens/model_time/cache_hit_ratioin the new "prompt complete" log are silently inflated by the summarization call — the exact "hidden spend from summarization" the plan flags. Fix: whenSummarizerModel == "", installcompaction.NewLLMSummarizerover the rawh.reg.LLM(ref)(ascompactionForalready does for the configured case) so the summarizer bypasses the clock wrapper.🟡 SUGGESTION —
internal/harness/harness.go:448: the preemption check runs exactly once, before the prompt knows it will be queued. A prompt arriving in the microsecond window between the running turn'sturnMu.Lock()andregisterTurnseesturnCancel == nil, no-ops, and then waits for the whole turn; a prompt queued behind an already-preempting prompt has its one-shot check spent and never preempts the turn that runs ahead of it. Both silently fall back to the pre-PR "wait for the whole turn" behavior. Fix: replace the boolean with a queued-prompt counter (increment at Prompt entry when a turn is running, decrement on registration) and have the running turn cancel at the next round boundary when the counter is non-zero.Question (not a finding): the clock block is appended as a trailing system-role content; the ADK Responses converter emits it in content order (
model/openaimodel/internal/responses/request.go:267), i.e. a system input item after user/tool items. OpenAI's Responses API requires system instructions first, so strict servers may reject or ignore it. The plan flags this as a "remaining risk" — please confirm the trailing system message was actually exercised against the real providers (Melious, local llama.cpp), since all e2e tests in this PR run against a fake server that accepts anything.@ -404,0 +445,4 @@// safe boundary: now when no tool call is in flight, otherwise when the// tool finishes. A tool call in flight is never severed; immediate// interruption remains session/cancel.h.requestPreemption()🟡 SUGGESTION — The preemption check runs exactly once, before this prompt knows whether it will actually be queued.
requestPreemption()executes beforeturnMu.Lock()(line 453) andregisterTurn(line 460). Two timings silently fall back to the pre-PR "wait for the whole turn" behavior: (a) a prompt arriving in the window between the running turn'sturnMu.Lock()andregisterTurnseesturnCancel == niland no-ops, then blocks for the entire turn; (b) a prompt queued behind an already-preempting prompt has its one-shot check already spent and never preempts the turn that runs ahead of it. Both are benign (no crash — old behavior) but make the Tier 4 guarantee timing-dependent.Fix: replace the
preemptPendingboolean with a queued-prompt counter — increment at Prompt entry whenever a turn is registered, decrement when a waiting prompt acquires the turn — and have the running turn cancel at the next round boundary when the counter is non-zero (still never while a tool is in flight).@ -296,0 +326,4 @@elapsed := time.Since(start)if m.clock != nil {m.clock.addUsage(usage, elapsed)🟠 IMPORTANT — Summarizer usage is accumulated into the running turn's clock when
compaction.summarizerModelis unset.This wrapper wraps every GenerateContent flowing through the root
selectorModel. With the default config (compaction on, 20k-token threshold, nosummarizerModel),compactionFor(internal/harness/harness.go:1200-1235) leavesout.Summarizernil, so ADK's runner installs its default LLMSummarizer over the root agent's model — thisselectorModel— (runner/runner.go:172-201:m := llminternal.Reveal(llmAgent).Model; NewLLMSummarizer(LLMSummarizerConfig{Model: m})). The summarizer callss.model.GenerateContent(...)directly (llm_summarizer.go:232), som.clock.addUsageadds the summarizer's prompt/cached tokens and call duration to the current turn, and the new "prompt complete" log reports inflatedprompt_tokens/cached_tokens/model_timeand a wrongcache_hit_ratiowhenever compaction fires mid-turn.Verified this session:
sed -n '172,201p' /home/paseo/go/pkg/mod/google.golang.org/adk/v2@v2.5.0/runner/runner.goandsed -n '232p' .../session/compaction/llm_summarizer.go(direct GenerateContent, no callbacks), plusgrep -n 'out.Summarizer' internal/harness/harness.go(compactionForsets it only whenSummarizerModel != "").Fix: when
SummarizerModel == "", installcompaction.NewLLMSummarizer(compaction.LLMSummarizerConfig{Model: llm})over the rawh.reg.LLM(ref)incompactionFor, mirroring the ADK default but bypassing this wrapper, so summarization usage never lands in the turn clock. (The configured-summarizer path already uses the raw llm and is unaffected.)Resolutions (head now
b08bd15)🟠 IMPORTANT — summarizer usage in the turn clock (comment 873): fixed.
compactionFornow always installs the summarizer over the raw registry model — the session's own model when nosummarizerModelis configured, mirroring what ADK would have defaulted to but routed past theselectorModelwrapper. Summarization calls can no longer land inm.clock.addUsage, soprompt_tokens/cached_tokens/model_time/cache_hit_ratioin theprompt completelog are the turn's own. The compaction test that previously assertedSummarizer == nilfor the unset case was updated to assert the new contract (non-nil, over the session model).🟡 SUGGESTION — one-shot preemption flag (comment 874): fixed.
preemptPending boolis replaced withqueuedPrompts int: every prompt increments at entry (before blocking on the turn lock) and retires its own entry viaacquiredTurnSlotonce it holds the slot. The running turn yields at the next safe boundary wheneverqueuedPrompts > 0, still never while a tool call is in flight. A prompt queued behind an already-preempting prompt now preempts the turn that runs ahead of it instead of silently waiting for it.Question — trailing system message against real providers: honest answer: not yet exercised against Melious or the local llama.cpp endpoint — every e2e test in this PR runs against a fake server that accepts anything, and I cannot reach either real provider from this container. The plan flags this as the remaining verification step. The concern is real (OpenAI's Responses API expects system instructions first; a strict server may reject or ignore a trailing system
input_text). Two ways forward, your call: (1) run one real prompt with the current build and watch whether the provider errors or the model still sees the clock block; (2) if it misbehaves, switch the clock to a trailing text part on the last user/tool content (role-user, accepted anywhere) — same cache property, slightly worse semantics (looks like a user message). I'd suggest verifying first before changing placement.Verification after both fixes:
go build, fullgo test ./...,racedetector test ./internal/harness/,golangci-lint— all clean.