fix: nudge the model back to work when a turn ends on prose #32

Merged
sam merged 4 commits from fix/nudge-prose-only-turns into main 2026-10-13 17:27:02 +00:00
Collaborator

Problem

Two independent causes produced the errors you kept seeing (adk run: openai: response output did not contain text or tool content) and made the retry solution look broken:

  1. The <brainless-clock> block was a mid-conversation role:"system" item. It was appended to every outgoing request inside input, after the user message. The Responses API only sanctions system text as the top-level instructions field or a leading input item. Melious's upstream silently drops a mid-conversation system item and answers HTTP 200 with nothing usable — its own error message says so: "Likely an unsupported request parameter that the provider silently dropped." Isolated with a live probe: instructions-only 4/4 usable, clock-only 0/4, plain 2/2. Pi never hits this because it never sends mid-conversation system messages (supportsMidConvoSystemMessages defaults false).

  2. The runner called the provider non-streaming. Brainless pinned StreamingModeNone; melious's non-streaming Responses path is the broken one (tool-bearing non-streaming requests came back with a reasoning item and no message 6/6 times in a probe), while streaming Responses returned usable turns 4/4 — the way pi drives melious.

Each degenerate round then burned the retry budget (30 × ~1s = ~30s of dead time per round, and the chat-completions fallback failed too because it inherited the same broken request shape).

Fix

  • Clock block is now a user-role content (request-only, never persisted, never visible to the user — verified 0 occurrences in streamed output). This removes the systematic degenerate-response trigger.
  • StreamingModeSSE: the provider is driven over SSE (the path it implements correctly), while the harness processes only the final aggregated event — the turn behaves exactly like a blocking call. The cache-write middleware now reads usage from the SSE terminal event, so write billing survives streaming.
  • Retry, retuned as defense in depth: retryEmptyOutput also retries ErrResponseFailed; when its budget on the Responses endpoint exhausts it tries a Chat Completions fallback for the same provider/model once (verified 6/6 against melious; both clients share the cache-write box).
  • Human-readable failures: exhaustion surfaces model X returned no usable output N times in a row (empty or failed provider responses)... instead of the raw ADK error; the original stays in the log.
  • Per-provider api config option ("responses" default | "chat_completions") for providers whose Responses endpoint is absent or broken.
  • Nudge now judges the final aggregated response of a stream, never an opening delta.
  • Memory results are now visible in the client timeline: the memory tools' outputs name the entities and quote the observations (no more count-only summaries), and the completed tool-call update for memory tools carries the result as a rendered text content block, so Paseo shows what the agent actually wrote to memory.
  • Fixes the long-standing varnamelen on NewRegistry — golangci-lint run ./... is now clean.

Verification

  • Live single-shot against melious before: retry storms of 24–30 attempts (~30s/round), turns failing with the raw error. After: no retry storms, wire roles all user, turns complete (6.5s typical for a single tool round; remaining latency is melious's ~77 tok/s × the rounds the model chooses).
  • Shape-isolation probe: instructions-only 4/4, clock-only 0/4, user-clock 6/6 usable.
  • e2e TestACPMemoryToolResultRendersContent: a create_memory result reaches the client as a rendered text content block naming the entity, type, and observation.
  • go test -race ./..., go vet ./... pass; golangci-lint run ./... 0 issues.
## Problem Two independent causes produced the errors you kept seeing (`adk run: openai: response output did not contain text or tool content`) and made the retry solution look broken: 1. **The `<brainless-clock>` block was a mid-conversation `role:"system"` item.** It was appended to every outgoing request inside `input`, *after* the user message. The Responses API only sanctions system text as the top-level `instructions` field or a *leading* input item. Melious's upstream silently drops a mid-conversation system item and answers HTTP 200 with nothing usable — its own error message says so: *"Likely an unsupported request parameter that the provider silently dropped."* Isolated with a live probe: `instructions`-only 4/4 usable, clock-only **0/4**, plain 2/2. Pi never hits this because it never sends mid-conversation system messages (`supportsMidConvoSystemMessages` defaults false). 2. **The runner called the provider non-streaming.** Brainless pinned `StreamingModeNone`; melious's non-streaming Responses path is the broken one (tool-bearing non-streaming requests came back with a reasoning item and no message **6/6** times in a probe), while **streaming** Responses returned usable turns 4/4 — the way pi drives melious. Each degenerate round then burned the retry budget (30 × ~1s = ~30s of dead time per round, and the chat-completions fallback failed too because it inherited the same broken request shape). ## Fix - **Clock block is now a user-role content** (request-only, never persisted, never visible to the user — verified 0 occurrences in streamed output). This removes the systematic degenerate-response trigger. - **`StreamingModeSSE`**: the provider is driven over SSE (the path it implements correctly), while the harness processes only the final aggregated event — the turn behaves exactly like a blocking call. The cache-write middleware now reads usage from the SSE terminal event, so write billing survives streaming. - **Retry, retuned as defense in depth**: `retryEmptyOutput` also retries `ErrResponseFailed`; when its budget on the Responses endpoint exhausts it tries a **Chat Completions fallback** for the same provider/model once (verified 6/6 against melious; both clients share the cache-write box). - **Human-readable failures**: exhaustion surfaces `model X returned no usable output N times in a row (empty or failed provider responses)...` instead of the raw ADK error; the original stays in the log. - **Per-provider `api` config option** (`"responses"` default | `"chat_completions"`) for providers whose Responses endpoint is absent or broken. - Nudge now judges the final aggregated response of a stream, never an opening delta. - **Memory results are now visible in the client timeline**: the memory tools' outputs name the entities and quote the observations (no more count-only summaries), and the completed tool-call update for memory tools carries the result as a rendered text content block, so Paseo shows what the agent actually wrote to memory. - Fixes the long-standing `varnamelen` on `NewRegistry` — `golangci-lint run ./...` is now clean. ## Verification - Live single-shot against melious before: retry storms of 24–30 attempts (~30s/round), turns failing with the raw error. After: **no retry storms, wire roles all `user`, turns complete** (6.5s typical for a single tool round; remaining latency is melious's ~77 tok/s × the rounds the model chooses). - Shape-isolation probe: instructions-only 4/4, clock-only 0/4, user-clock 6/6 usable. - e2e `TestACPMemoryToolResultRendersContent`: a `create_memory` result reaches the client as a rendered text content block naming the entity, type, and observation. - `go test -race ./...`, `go vet ./...` pass; `golangci-lint run ./...` 0 issues.
fix: nudge the model back to work when a turn ends on prose
All checks were successful
code-review / Agent review (pull_request) Successful in 15s
c52065e3c3
ADK's flow ends a run the moment the model answers with text and no tool
call, so a prose-only turn executes nothing: the client sees the text, the
turn dies, and the next prompt re-sends the whole history (~521k tokens per
task) for zero progress. The model knows it should call tools; the
difference is whether a call is physically in the message.

Wrap every registry client in nudgeToolUse (outside retryEmptyOutput). When
a response carries text and no tool call while the request shows no tool
work since the last user text message, discard it and re-send with a firm
continuation nudge appended as a user message, up to toolUseNudgeMax times.
The nudge only fires when the request carries tools (the compaction
summarizer's tool-less summarization call is never nudged) and never for
thought-only or tool-call responses; a text-only answer after tool work is
a legitimate final answer and passes through untouched.
pi-reviewer left a comment

Review of fix/nudge-prose-only-turns at c52065e, verified against the repo's own toolchain and the ADK v2.5.0 / genai v1.72.0 module sources.

Verified green (commands run in a worktree at the PR head): go build ./... OK; go vet ./... OK; go test -race ./internal/harness/ OK — all 7 new unit tests plus TestACPNudgeKeepsTurnAliveAcrossProseOnlyAnswer pass; golangci-lint run ./internal/harness/ clean except the disclosed pre-existing varnamelen on NewRegistry; no go.mod/go.sum changes in the diff (no new dependencies; govulncheck reports only pre-existing, uncalled vulnerabilities).

API claims checked against module source, not memory: model.LLM/LLMRequest shape (go doc google.golang.org/adk/v2/model.LLM); genai.Content.Parts/Role, Part.Text/Thought/FunctionCall/FunctionResponse, FunctionCall.Name/Args, FunctionResponse.Name/Response; ADK's base flow appends tool results as user content with FunctionResponse parts and no text part (internal/llminternal/base_flow.go), so hasToolWorkThisTurn matches the real conversation shape; ADK itself re-calls thought-only turns (maxConsecutiveThoughtOnlyTurns = 10, base_flow.go), so the thought-only pass-through is safe.

No blocking issues. One IMPORTANT finding and two suggestions below.

IMPORTANT — the retry and nudge budgets compose multiplicatively (registry.go:99-100): each nudge round calls retryEmptyOutput.GenerateContent fresh, re-arming its own 30-attempt / 15s budget, so a model that alternates empty outputs and prose-only answers (both failure modes documented for melious/deepseek-v4-flash-0731, the exact model this PR targets) can cost up to 3 × 31 = 93 full-conversation re-sends and ~45s per model call. Suggest sharing one budget across both wrappers before merge.

Review of fix/nudge-prose-only-turns at c52065e, verified against the repo's own toolchain and the ADK v2.5.0 / genai v1.72.0 module sources. **Verified green** (commands run in a worktree at the PR head): `go build ./...` OK; `go vet ./...` OK; `go test -race ./internal/harness/` OK — all 7 new unit tests plus `TestACPNudgeKeepsTurnAliveAcrossProseOnlyAnswer` pass; `golangci-lint run ./internal/harness/` clean except the disclosed pre-existing `varnamelen` on `NewRegistry`; no go.mod/go.sum changes in the diff (no new dependencies; govulncheck reports only pre-existing, uncalled vulnerabilities). **API claims checked against module source, not memory**: `model.LLM`/`LLMRequest` shape (`go doc google.golang.org/adk/v2/model.LLM`); `genai.Content.Parts/Role`, `Part.Text/Thought/FunctionCall/FunctionResponse`, `FunctionCall.Name/Args`, `FunctionResponse.Name/Response`; ADK's base flow appends tool results as user content with `FunctionResponse` parts and no text part (`internal/llminternal/base_flow.go`), so `hasToolWorkThisTurn` matches the real conversation shape; ADK itself re-calls thought-only turns (`maxConsecutiveThoughtOnlyTurns = 10`, base_flow.go), so the thought-only pass-through is safe. **No blocking issues.** One IMPORTANT finding and two suggestions below. IMPORTANT — the retry and nudge budgets compose multiplicatively (registry.go:99-100): each nudge round calls `retryEmptyOutput.GenerateContent` fresh, re-arming its own 30-attempt / 15s budget, so a model that alternates empty outputs and prose-only answers (both failure modes documented for melious/deepseek-v4-flash-0731, the exact model this PR targets) can cost up to 3 × 31 = 93 full-conversation re-sends and ~45s per model call. Suggest sharing one budget across both wrappers before merge.
@ -661,0 +760,4 @@
// The nudge must not end the turn: the tool the model called on the second
// round executed (start + completed), so the turn produced real work.
if len(client.toolCalls) != 2 ||
Collaborator

🟡 SUGGESTION — the test cannot detect a wrong nudge on the final answer. The stub serves the same text ("Found it: pong.") for every request >= 3 (default case, lines 699-703), and the assertions here (2 tool calls, updates = [Let me look at the file., Found it: pong.]) are identical whether the final answer passed through cleanly or was wrongly nudged twice (the model would answer the same text again, then the budget forces a pass-through). So a regression where hasToolWorkThisTurn always returns false still passes this e2e. Verified: stub + assertions at c52065e; test passes with current code. Fix: count requests in the stub (it already tracks step) and assert exactly 3 — that pins the "final answer after tool work is never nudged" claim to the real flow instead of the hand-built shape in TestNudgeToolUseSkipsWhenToolsRanThisTurn.

🟡 SUGGESTION — the test cannot detect a wrong nudge on the final answer. The stub serves the same text ("Found it: pong.") for every request >= 3 (default case, lines 699-703), and the assertions here (2 tool calls, updates = [Let me look at the file., Found it: pong.]) are identical whether the final answer passed through cleanly or was wrongly nudged twice (the model would answer the same text again, then the budget forces a pass-through). So a regression where `hasToolWorkThisTurn` always returns false still passes this e2e. Verified: stub + assertions at c52065e; test passes with current code. Fix: count requests in the stub (it already tracks `step`) and assert exactly 3 — that pins the "final answer after tool work is never nudged" claim to the real flow instead of the hand-built shape in TestNudgeToolUseSkipsWhenToolsRanThisTurn.
@ -0,0 +56,4 @@
func (n *nudgeToolUse) Name() string { return n.llm.Name() }
func (n *nudgeToolUse) GenerateContent(ctx context.Context, req *model.LLMRequest, stream bool) iter.Seq2[*model.LLMResponse, error] {
Collaborator

🟡 SUGGESTION — the wrapper ignores the stream flag. Under streaming, ADK's openaimodel "yield[s] a partial per delta and one final response built from the accumulated turn" (internal/completions/model.go, generateStream), so the first pair is a partial text delta with no tool call yet: eligibleForToolUseNudge would fire on it, the whole accumulated stream would be discarded and re-sent, and collectModelPairs buffers the entire stream in memory. Today unreachable — the only call site pins agent.StreamingModeNone (harness.go:724, verified) and registry.go:379-380 documents the same caveat for cacheWriteMiddleware — but a one-line guard (pass pairs through unchanged when stream == true) keeps this wrapper streaming-safe for when that path appears.

🟡 SUGGESTION — the wrapper ignores the `stream` flag. Under streaming, ADK's openaimodel "yield[s] a partial per delta and one final response built from the accumulated turn" (internal/completions/model.go, generateStream), so the first pair is a partial text delta with no tool call yet: `eligibleForToolUseNudge` would fire on it, the whole accumulated stream would be discarded and re-sent, and `collectModelPairs` buffers the entire stream in memory. Today unreachable — the only call site pins `agent.StreamingModeNone` (harness.go:724, verified) and registry.go:379-380 documents the same caveat for cacheWriteMiddleware — but a one-line guard (pass pairs through unchanged when `stream == true`) keeps this wrapper streaming-safe for when that path appears.
@ -95,6 +97,7 @@ func NewRegistry(ctx context.Context, f *config.File) (*Registry, error) {
}
llm = &retryEmptyOutput{llm: llm, delay: emptyOutputRetryDelay}
llm = &nudgeToolUse{llm: llm, delay: toolUseNudgeDelay}
Collaborator

🟠 IMPORTANT — the two wrappers' budgets compose multiplicatively. nudgeToolUse wraps retryEmptyOutput, and each nudge round (up to toolUseNudgeMax = 2, so 3 calls total) calls retryEmptyOutput.GenerateContent fresh, re-arming its own budget: maxEmptyOutputRetries = 30 attempts / emptyOutputRetryBudget = 15 * time.Second per call (retry.go, verified at c52065e). Worst case per flow model call: 3 × 31 = 93 full-conversation re-sends and ~45s — each re-send costs the whole conversation (~521k tokens/turn per the PR description). The condition is not exotic: retry.go documents the target model (melious/deepseek-v4-flash-0731) answering empty ~2/3 of the time, and this PR exists because the same model answers prose-only. go build ./... passes; the wiring is llm = &retryEmptyOutput{...}; llm = &nudgeToolUse{llm: llm, ...} (lines 99-100). Fix: share one budget across both wrappers — pass a shared attempt counter / time budget into the chain, or count nudge re-sends against the same 15s/30-attempt budget instead of each round restarting it.

🟠 IMPORTANT — the two wrappers' budgets compose multiplicatively. `nudgeToolUse` wraps `retryEmptyOutput`, and each nudge round (up to `toolUseNudgeMax` = 2, so 3 calls total) calls `retryEmptyOutput.GenerateContent` fresh, re-arming its own budget: `maxEmptyOutputRetries = 30` attempts / `emptyOutputRetryBudget = 15 * time.Second` per call (retry.go, verified at c52065e). Worst case per flow model call: 3 × 31 = 93 full-conversation re-sends and ~45s — each re-send costs the whole conversation (~521k tokens/turn per the PR description). The condition is not exotic: retry.go documents the target model (melious/deepseek-v4-flash-0731) answering empty ~2/3 of the time, and this PR exists because the same model answers prose-only. `go build ./...` passes; the wiring is `llm = &retryEmptyOutput{...}; llm = &nudgeToolUse{llm: llm, ...}` (lines 99-100). Fix: share one budget across both wrappers — pass a shared attempt counter / time budget into the chain, or count nudge re-sends against the same 15s/30-attempt budget instead of each round restarting it.
Two independent causes produced the empty-output errors and the retry
storms behind them ("adk run: openai: response output did not contain text
or tool content").

1. The <brainless-clock> block was appended to every outgoing request as a
   role:"system" item inside input, after the user message. The Responses
   API only sanctions system text as the top-level instructions field or a
   leading input item; melious's upstream silently drops a mid-conversation
   system item and answers HTTP 200 with nothing usable (its own error:
   "unsupported request parameter that the provider silently dropped").
   Every round then hit the degenerate path. A probe isolating the shapes
   confirmed it: instructions-only 4/4 usable, clock-only 0/4, plain 2/2.
   The block is now a user-role content; it stays request-only (never
   persisted, never visible to the client), so the user never sees it.

2. The runner called the provider non-streaming (StreamingModeNone), the
   path melious implements worst: tool-bearing non-streaming requests came
   back with a reasoning item and no message 6/6 times in a probe, while
   streaming Responses returned usable turns 4/4 (pi drives melious the
   same way). The harness now runs StreamingModeSSE and processes only the
   final aggregated event, so the turn behaves exactly like a blocking
   call. The cache-write middleware now parses the SSE body's terminal
   usage event, so write billing survives streaming.

Defense in depth stays in place, retuned:
- retryEmptyOutput also retries ErrResponseFailed, and when its budget on
  the Responses endpoint exhausts it tries a Chat Completions client for
  the same provider/model once (both share the cache-write box).
- Exhaustion now surfaces a human-readable error ("model X returned no
  usable output N times in a row...") instead of the raw ADK error, with
  the original kept in the log.
- A per-provider api config option ("responses" default |
  "chat_completions") lets an operator pin a provider to Chat Completions.
- The nudge (text-only stops without tool work) now judges the final
  aggregated response of a stream, never an opening delta.

Also fixes the long-standing varnamelen on NewRegistry, so golangci-lint is
clean.
feat: show what the agent wrote to memory in the client timeline
All checks were successful
code-review / Agent review (pull_request) Successful in 8s
0740791c87
The memory tools summarized their work as counts ("Created 2 entities, 1
relation."), so a client rendering the tool-call result showed nothing
about the actual memory content. Two changes make it visible:

- The tool results now carry the content: create_memory names each created
  entity (with its type and quoted observations) and each relation triple;
  add_observations names the entity and quotes the added observations;
  delete_memory names the deleted entities.
- The completed tool-call update for memory tools (create_memory,
  add_observations, delete_memory, read_graph, load_memory, preload_memory)
  now also carries the result text as a standard content block, the ACP
  field clients render in the timeline, alongside rawOutput.

Tests: unit test on the enriched tool outputs; e2e test proving a
create_memory result reaches the client as a rendered text content block.
pi-reviewer left a comment

Review of PR #32 (fix/nudge-prose-only-turns), verified against head 0740791 in a fresh worktree (Go 1.27.1, ADK v2.5.0).

What holds up:

  • go vet ./... clean; go test ./... and go test -race ./internal/harness/ green at head; the PR's own tests pass.
  • API claims checked against the module cache: session.Event embeds model.LLMResponse so ev.Partial exists (session/session.go:100-101, model/llm.go:56); openaimodel.ErrResponseFailed/ErrNoTextOrToolContent/ErrNoOutputItems/ErrNoChoices exist (model/openaimodel/errors.go); ClientConfig.API + APIChatCompletions exist (model/openaimodel/openaimodel.go:31-41). No go.mod/go.sum changes.
  • The user-role clock change and the SSE streaming switch address the reported provider shape; the memory-result content blocks render (TestACPMemoryToolResultRendersContent passes).

One BLOCKING finding: the new nudge guard hasToolWorkThisTurn (nudge.go:198) is defeated by the turn-clock block, which this very PR turns into a user-role text content appended to every request (harness.go:315-323, run before the model call per ADK v2.5.0 internal/llminternal/base_flow.go:957-966). Reproduced with a temp e2e test against the real harness: a turn that should complete in 2 model calls makes 4, with two nudging it to continue WARNs fired on the final answer. Every text-only final answer after tool work is discarded and re-sent with the nudge (up to 2×) — extra full-conversation re-sends, extra cost, and the model is told to call tools after the task is done. Evidence is in the inline comment; the PR's e2e stub serves the same text for every request ≥ 2, so its test cannot catch this.

Verdict: REQUEST_CHANGES (1 blocking + 5 suggestions).

Review of PR #32 (fix/nudge-prose-only-turns), verified against head 0740791 in a fresh worktree (Go 1.27.1, ADK v2.5.0). What holds up: - `go vet ./...` clean; `go test ./...` and `go test -race ./internal/harness/` green at head; the PR's own tests pass. - API claims checked against the module cache: `session.Event` embeds `model.LLMResponse` so `ev.Partial` exists (session/session.go:100-101, model/llm.go:56); `openaimodel.ErrResponseFailed`/`ErrNoTextOrToolContent`/`ErrNoOutputItems`/`ErrNoChoices` exist (model/openaimodel/errors.go); `ClientConfig.API` + `APIChatCompletions` exist (model/openaimodel/openaimodel.go:31-41). No go.mod/go.sum changes. - The user-role clock change and the SSE streaming switch address the reported provider shape; the memory-result content blocks render (TestACPMemoryToolResultRendersContent passes). One BLOCKING finding: the new nudge guard `hasToolWorkThisTurn` (nudge.go:198) is defeated by the turn-clock block, which this very PR turns into a user-role text content appended to every request (harness.go:315-323, run before the model call per ADK v2.5.0 internal/llminternal/base_flow.go:957-966). Reproduced with a temp e2e test against the real harness: a turn that should complete in 2 model calls makes 4, with two `nudging it to continue` WARNs fired on the final answer. Every text-only final answer after tool work is discarded and re-sent with the nudge (up to 2×) — extra full-conversation re-sends, extra cost, and the model is told to call tools after the task is done. Evidence is in the inline comment; the PR's e2e stub serves the same text for every request ≥ 2, so its test cannot catch this. Verdict: REQUEST_CHANGES (1 blocking + 5 suggestions).
@ -661,0 +810,4 @@
// The discarded first prose is never streamed; only the nudged round's
// narrative and the final answer reach the client.
wantUpdates := []string{"Let me look at the file.", "Found it: pong."}
Collaborator

🟡 SUGGESTION — the stub's default branch of newProseThenToolResponsesServer serves the same final text for every request ≥ 2, so this test passes even when the nudge misfires on the final answer: with the guard broken the harness still produces the same client.updates (4 model calls; the discarded answers are never streamed). That is exactly what my failing e2e run produced. Add an assertion on the total request count (== 3) so a guard regression fails this test loudly.

🟡 SUGGESTION — the stub's `default` branch of newProseThenToolResponsesServer serves the same final text for every request ≥ 2, so this test passes even when the nudge misfires on the final answer: with the guard broken the harness still produces the same `client.updates` (4 model calls; the discarded answers are never streamed). That is exactly what my failing e2e run produced. Add an assertion on the total request count (== 3) so a guard regression fails this test loudly.
@ -1197,1 +1206,4 @@
// unknown call into "other".
// isMemoryTool reports whether name is one of the memory curation/search
// tools, whose results the client should render as text.
func (h *Harness) isMemoryTool(name string) bool {
Collaborator

🟡 SUGGESTION — isMemoryTool is inserted between toolKind's doc comment (lines 1199-1206) and the toolKind function (line 1217), so the doc comment now documents isMemoryTool — which it does not describe — and toolKind is left undocumented. Move isMemoryTool below toolKind.

🟡 SUGGESTION — `isMemoryTool` is inserted between `toolKind`'s doc comment (lines 1199-1206) and the `toolKind` function (line 1217), so the doc comment now documents `isMemoryTool` — which it does not describe — and `toolKind` is left undocumented. Move `isMemoryTool` below `toolKind`.
@ -0,0 +195,4 @@
// exactly this turn's activity: a tool round appends the model's function
// calls and the tool results, so a text-only response after them is a final
// answer, not a premature stop.
func hasToolWorkThisTurn(contents []*genai.Content) bool {
Collaborator

🔴 BLOCKING — hasToolWorkThisTurn is defeated by the turn-clock block, so the nudge misfires on legitimate final answers after tool work.

Reproduced against the PR head with temp tests in the harness package:

  • go test ./internal/harness/ -run TestReviewClockBlockDefeatsToolWorkGuard → FAIL: hasToolWorkThisTurn = false for a conversation containing a function call + function response, because the appended user-role clock block (<brainless-clock>...) is the last user-role TEXT content and the tail after it is empty.
  • go test ./internal/harness/ -run TestReviewFinalAnswerAfterToolWorkNotNudged → FAIL: model called 4 times, want 2 with two WARN model answered with text and no tool call; nudging it to continue lines fired on the final answer.

Why: harness.go:315-323 appends the clock block as a user-role text content to every outgoing request (ADK v2.5.0 internal/llminternal/base_flow.go:957-966 runs BeforeModelCallbacks before m.GenerateContent), so this loop always lands on the clock block as the "last user text message" and contents[lastUserText+1:] is empty. The guard !hasToolWorkThisTurn(req.Contents) in eligibleForToolUseNudge (line 144) is therefore always true in production; TestNudgeToolUseSkipsWhenToolsRanThisTurn passes only because its fixture has no clock block.

Impact: every turn ending with a text-only final answer after tool work gets that answer discarded and re-sent with the nudge up to 2× — two extra full-conversation model calls per turn (latency + prefill cost) — and the model is instructed to call tools after the task is done. This is exactly the case the guard's own doc comment says must pass through untouched.

Fix: skip the clock block in the scan (ignore user-role contents whose text starts with <brainless-clock>), or anchor the "last user text" scan before the trailing clock block.

🔴 BLOCKING — `hasToolWorkThisTurn` is defeated by the turn-clock block, so the nudge misfires on legitimate final answers after tool work. Reproduced against the PR head with temp tests in the harness package: - `go test ./internal/harness/ -run TestReviewClockBlockDefeatsToolWorkGuard` → FAIL: `hasToolWorkThisTurn = false` for a conversation containing a function call + function response, because the appended user-role clock block (`<brainless-clock>...`) is the last user-role TEXT content and the tail after it is empty. - `go test ./internal/harness/ -run TestReviewFinalAnswerAfterToolWorkNotNudged` → FAIL: `model called 4 times, want 2` with two `WARN model answered with text and no tool call; nudging it to continue` lines fired on the final answer. Why: harness.go:315-323 appends the clock block as a user-role text content to every outgoing request (ADK v2.5.0 internal/llminternal/base_flow.go:957-966 runs BeforeModelCallbacks before `m.GenerateContent`), so this loop always lands on the clock block as the "last user text message" and `contents[lastUserText+1:]` is empty. The guard `!hasToolWorkThisTurn(req.Contents)` in eligibleForToolUseNudge (line 144) is therefore always true in production; TestNudgeToolUseSkipsWhenToolsRanThisTurn passes only because its fixture has no clock block. Impact: every turn ending with a text-only final answer after tool work gets that answer discarded and re-sent with the nudge up to 2× — two extra full-conversation model calls per turn (latency + prefill cost) — and the model is instructed to call tools after the task is done. This is exactly the case the guard's own doc comment says must pass through untouched. Fix: skip the clock block in the scan (ignore user-role contents whose text starts with `<brainless-clock>`), or anchor the "last user text" scan before the trailing clock block.
Collaborator

🟡 SUGGESTION — with StreamingModeSSE now always on (harness.go:724), this middleware's io.ReadAll buffers the entire SSE stream (all deltas + the terminal event) before the client decodes anything: every Responses response is fully materialized in memory and the streaming client gains no incremental delivery at the HTTP layer. Fine for typical answers; worth a cap or a streaming read if long streams become common.

🟡 SUGGESTION — with StreamingModeSSE now always on (harness.go:724), this middleware's `io.ReadAll` buffers the entire SSE stream (all deltas + the terminal event) before the client decodes anything: every Responses response is fully materialized in memory and the streaming client gains no incremental delivery at the HTTP layer. Fine for typical answers; worth a cap or a streaming read if long streams become common.
@ -126,0 +160,4 @@
slog.String("model", r.llm.Name()),
slog.Int("attempts", attempts))
fb := (&retryEmptyOutput{llm: r.fallback, delay: r.delay}).GenerateContent(ctx, req, stream)
Collaborator

🟡 SUGGESTION — the fallback is double-wrapped: NewRegistry already wraps the chat-completions client in retryEmptyOutput (registry.go:134), and this line wraps it again, re-arming a full 15s/30-attempt retry budget on top of the exhausted primary budget — worst case ~30s of degenerate re-sends per model call before the human-readable failure. Call r.fallback.GenerateContent(ctx, req, stream) directly; the registry wrapper already provides the retry.

Also, fbPairs[0].err != nil only inspects the first pair: a streamed fallback that yields deltas and then fails surfaces the raw ADK error instead of the human-readable message. Check the final pair.

🟡 SUGGESTION — the fallback is double-wrapped: NewRegistry already wraps the chat-completions client in `retryEmptyOutput` (registry.go:134), and this line wraps it again, re-arming a full 15s/30-attempt retry budget on top of the exhausted primary budget — worst case ~30s of degenerate re-sends per model call before the human-readable failure. Call `r.fallback.GenerateContent(ctx, req, stream)` directly; the registry wrapper already provides the retry. Also, `fbPairs[0].err != nil` only inspects the first pair: a streamed fallback that yields deltas and then fails surfaces the raw ADK error instead of the human-readable message. Check the final pair.
@ -114,3 +114,3 @@
if got := req1.Contents[2].Role; got != "system" {
if got := req1.Contents[2].Role; got != "user" {
t.Fatalf("block role = %q, want system", got)
Collaborator

🟡 SUGGESTION — the assertion now expects role "user" but the failure message still says want system; update the message.

🟡 SUGGESTION — the assertion now expects role "user" but the failure message still says `want system`; update the message.
pi-agent referenced this pull request from a commit 2026-10-13 17:17:24 +00:00
BLOCKING: the user-role <brainless-clock> block defeated the nudge's
tool-work guard — it is the last user-role text content of every outgoing
request, so the tail scan after it was always empty and every text-only
final answer after tool work was discarded and re-sent with the nudge (two
extra full-conversation calls per turn). hasToolWorkThisTurn now skips
clock blocks when finding the turn's last user message; a regression test
covers a tool round with a trailing clock block, and the prose-stop e2e
now asserts exactly 3 model calls so a guard regression fails loudly.

SUGGESTIONS:
- retry and nudge budgets now share one per-call degenerate-attempt budget
  (carried in the context the nudge installs), so they cannot multiply
  (was up to 3 x 31 = 93 re-sends/~45s per model call; now bounded by one
  30-attempt/15s cap); a composition test pins the bound.
- the chat-completions fallback is no longer double-wrapped (the registry
  already wraps it), and the fallback failure check reads the final pair so
  a streamed fallback that fails surfaces the human-readable message; the
  fallback gets its own fresh budget.
- the cache-write middleware buffers the SSE body under a 64 MiB cap
  instead of unbounded io.ReadAll.
- isMemoryTool moved below toolKind so toolKind keeps its doc comment; the
  turn-clock test's failure message now says 'want user'.
Author
Collaborator

Resolution record for reviews 58 (c52065e) and 59 (0740791) — fixed at head 6e89597, go test -race ./... + go vet ./... green, golangci-lint run ./... 0 issues.

🔴 1087 (BLOCKING — clock block defeats the tool-work guard): fixed. hasToolWorkThisTurn now skips user-role contents whose text starts with <brainless-clock> when locating the turn's last user message, so the guard sees the real conversation tail again. Regression tests: TestNudgeToolUseSkipsWhenClockBlockTrailsToolWork (tool round + trailing clock block passes through untouched) and the e2e now asserts exactly 3 model calls.

🟡 1088 / 1081 (e2e cannot detect a wrong nudge on the final answer): fixed. newProseThenToolResponsesServer exposes a request counter; TestACPNudgeKeepsTurnAliveAcrossProseOnlyAnswer asserts requests() == 3, so a guard regression that nudges the final answer fails loudly.

🟡 1089 (isMemoryTool stole toolKind's doc comment): fixed. isMemoryTool moved below toolKind; toolKind's doc comment documents toolKind again.

🟡 1090 (stale 'want system' message): fixed. The turn-clock test now expects role "user" in its failure message.

🟡 1091 (fallback double-wrapped + first-pair check): fixed. fallbackOrGiveUp calls r.fallback.GenerateContent directly (the registry already wraps the chat-completions client in its own retry), the failure check reads the final pair so a streamed fallback that fails surfaces the human-readable message, and the fallback gets a fresh budget (the shared one is exhausted).

🟡 1092 (unbounded middleware buffering): fixed. The cache-write middleware reads the body under a 64 MiB cap (maxResponseBodyBytes) and fails the call clearly over it instead of holding unbounded memory.

🟠 1079 (retry × nudge budgets compose multiplicatively): fixed. The retry and nudge wrappers now share one per-call degenerate-attempt budget carried in the context the nudge installs (retryBudget/budgetFor/spend), so the worst case is one 30-attempt/15s cap instead of 3 × 31 = 93 re-sends. TestNudgeAndRetryShareOneBudget pins the bound: an alternating prose/degenerate provider stops after 33 attempts (was 43+ unshared).

🟡 1080 (stream flag ignored): resolved by the streaming work in 5f423e4 — the harness now runs StreamingModeSSE, and the nudge decides on the final non-partial response (finalResponse), so partial deltas pass through and never trigger a nudge; both wrappers pass stream through unchanged.

Resolution record for reviews 58 (c52065e) and 59 (0740791) — fixed at head 6e89597, `go test -race ./...` + `go vet ./...` green, `golangci-lint run ./...` 0 issues. **🔴 1087 (BLOCKING — clock block defeats the tool-work guard)**: fixed. `hasToolWorkThisTurn` now skips user-role contents whose text starts with `<brainless-clock>` when locating the turn's last user message, so the guard sees the real conversation tail again. Regression tests: `TestNudgeToolUseSkipsWhenClockBlockTrailsToolWork` (tool round + trailing clock block passes through untouched) and the e2e now asserts exactly 3 model calls. **🟡 1088 / 1081 (e2e cannot detect a wrong nudge on the final answer)**: fixed. `newProseThenToolResponsesServer` exposes a request counter; `TestACPNudgeKeepsTurnAliveAcrossProseOnlyAnswer` asserts `requests() == 3`, so a guard regression that nudges the final answer fails loudly. **🟡 1089 (isMemoryTool stole toolKind's doc comment)**: fixed. `isMemoryTool` moved below `toolKind`; `toolKind`'s doc comment documents `toolKind` again. **🟡 1090 (stale 'want system' message)**: fixed. The turn-clock test now expects role "user" in its failure message. **🟡 1091 (fallback double-wrapped + first-pair check)**: fixed. `fallbackOrGiveUp` calls `r.fallback.GenerateContent` directly (the registry already wraps the chat-completions client in its own retry), the failure check reads the final pair so a streamed fallback that fails surfaces the human-readable message, and the fallback gets a fresh budget (the shared one is exhausted). **🟡 1092 (unbounded middleware buffering)**: fixed. The cache-write middleware reads the body under a 64 MiB cap (`maxResponseBodyBytes`) and fails the call clearly over it instead of holding unbounded memory. **🟠 1079 (retry × nudge budgets compose multiplicatively)**: fixed. The retry and nudge wrappers now share one per-call degenerate-attempt budget carried in the context the nudge installs (`retryBudget`/`budgetFor`/`spend`), so the worst case is one 30-attempt/15s cap instead of 3 × 31 = 93 re-sends. `TestNudgeAndRetryShareOneBudget` pins the bound: an alternating prose/degenerate provider stops after 33 attempts (was 43+ unshared). **🟡 1080 (stream flag ignored)**: resolved by the streaming work in 5f423e4 — the harness now runs StreamingModeSSE, and the nudge decides on the final non-partial response (`finalResponse`), so partial deltas pass through and never trigger a nudge; both wrappers pass `stream` through unchanged.
sam merged commit 4a8823af22 into main 2026-10-13 17:27:02 +00:00
sam deleted branch fix/nudge-prose-only-turns 2026-10-13 17:27:02 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!32
No description provided.