fix: retry model responses that carry no text or tool content #29

Merged
sam merged 2 commits from fix/retry-empty-model-response into main 2026-10-12 07:14:22 +00:00
Collaborator

Problem

Brainless turns in Paseo fail intermittently with:

Internal error | code=-32603 | data={"error":"adk run: openai: response output did not contain text or tool content"}

Root cause: the Google ADK v2.5.0 blocking Responses path
(model/openaimodel/internal/responses/response.go, convertOutputItems) fails
the whole turn when a provider response has output items but none of them
convert to a text or tool-call part — all reasoning/message items arrive
with empty text. Melious intermittently returns exactly that shape for the
reasoning models brainless runs (seen on melious/deepseek-v4-flash-0731 in
the Oct 11 failure; sibling pi agent on the same model ran fine, and ~35
direct API calls reproduced no degenerate shape — a transient provider
response). ADK's streaming path tolerates empty outputs; brainless runs
StreamingModeNone, so the blocking path fails hard.

Fix

Wrap every registry model client (NewRegistry) in a retryEmptyOutput
decorator (internal/harness/retry.go):

  • Re-sends the identical request up to 2 more times (200 ms apart) when the
    only thing yielded is openaimodel.ErrNoOutputItems /
    openaimodel.ErrNoTextOrToolContent.
  • Retrying is safe: an empty response executed no tool calls, so a retry
    cannot double-run anything.
  • A call that produced at least one response is never retried; a trailing
    error after yielded responses is preserved.
  • The compaction summarizer routes through the same wrapped client, so
    summarization gets the same protection.

Tests: internal/harness/retry_test.go (retry-once, exhaust-retries,
no-retry-on-other-errors, preserve-trailing-error). go test ./...,
go test -race ./internal/harness/, and golangci-lint run ./... all pass
(the only lint finding is a pre-existing varnamelen on the untouched
NewRegistry signature).

Note on streaming: switching the runner to streaming would hide this bug
(ADK streams degenerate responses as an empty successful turn), not fix it —
the user would get no answer and no error. Retry actively recovers; streaming
is a separate UX change.

## Problem Brainless turns in Paseo fail intermittently with: ``` Internal error | code=-32603 | data={"error":"adk run: openai: response output did not contain text or tool content"} ``` Root cause: the Google ADK v2.5.0 **blocking** Responses path (`model/openaimodel/internal/responses/response.go`, `convertOutputItems`) fails the whole turn when a provider response has output items but none of them convert to a text or tool-call part — all `reasoning`/`message` items arrive with empty text. Melious intermittently returns exactly that shape for the reasoning models brainless runs (seen on `melious/deepseek-v4-flash-0731` in the Oct 11 failure; sibling pi agent on the same model ran fine, and ~35 direct API calls reproduced no degenerate shape — a transient provider response). ADK's streaming path tolerates empty outputs; brainless runs `StreamingModeNone`, so the blocking path fails hard. ## Fix Wrap every registry model client (`NewRegistry`) in a `retryEmptyOutput` decorator (`internal/harness/retry.go`): - Re-sends the identical request up to 2 more times (200 ms apart) when the only thing yielded is `openaimodel.ErrNoOutputItems` / `openaimodel.ErrNoTextOrToolContent`. - Retrying is safe: an empty response executed no tool calls, so a retry cannot double-run anything. - A call that produced at least one response is never retried; a trailing error after yielded responses is preserved. - The compaction summarizer routes through the same wrapped client, so summarization gets the same protection. Tests: `internal/harness/retry_test.go` (retry-once, exhaust-retries, no-retry-on-other-errors, preserve-trailing-error). `go test ./...`, `go test -race ./internal/harness/`, and `golangci-lint run ./...` all pass (the only lint finding is a pre-existing `varnamelen` on the untouched `NewRegistry` signature). Note on streaming: switching the runner to streaming would hide this bug (ADK streams degenerate responses as an empty successful turn), not fix it — the user would get no answer and no error. Retry actively recovers; streaming is a separate UX change.
ADK's blocking Responses path fails the whole turn when a provider
answers with output that converts to no parts (reasoning or message
items with empty text). Melious intermittently returns exactly that
for the reasoning models brainless runs, surfacing in Paseo as:

  adk run: openai: response output did not contain text or tool content

Wrap every registry client in a retry that re-sends a degenerate
response up to twice before surfacing the error. Retrying the same
request is safe: an empty response executed no tool calls. A call
that produced at least one response is never retried, and a trailing
error after yielded responses still reaches the consumer. The
compaction summarizer routes through the same wrapped client, so
summarization gets the same protection.
pi-agent force-pushed fix/retry-empty-model-response from 60a00f6a23 to 178fea0e25
All checks were successful
code-review / Agent review (pull_request) Successful in 8s
2026-10-11 20:43:30 +00:00
Compare
pi-reviewer left a comment

Review: fix/retry-empty-model-response

Retry-on-empty-output decorator for the ADK openaimodel client. Reviewed against a worktree at PR head 178fea0 (go1.27.1 toolchain, module go 1.27.1).

Verification performed in this session

  • go build ./... → clean; go vet ./internal/harness/ → clean; go test -count=1 ./... → all packages pass; go test -race -count=1 ./internal/harness/ → ok (8.4s).
  • golangci-lint run ./internal/harness/... → only the pre-existing varnamelen on NewRegistry's f parameter, exactly as the PR description states.
  • ADK claims confirmed against google.golang.org/adk/v2@v2.5.0 module-cache source: openaimodel.ErrNoOutputItems/ErrNoTextOrToolContent exist (model/openaimodel/errors.go:56,64) as errors.New sentinels (internal/shared/errors.go:54,67), so the errors.Is matching in emptyOutputError is correct. model.LLM is { Name() string; GenerateContent(ctx, req, stream) iter.Seq2[*LLMResponse, error] }; the decorator satisfies it.
  • Retry-trigger semantics match the real blocking path (internal/responses/model.go:49-53): a degenerate conversion yields exactly one (nil, err) pair, so the wrapper's "single pair is an empty-output error → retry" logic is exact. On both the Responses and Chat Completions paths, a tool call always appends a part before any empty-output error can fire (convertOutputItems / convertMessage), so the PR's safety claim "an empty response executed no tool calls" holds — the retry cannot double-run a tool.

Findings

No blocking or important issues. Three suggestions, all in internal/harness/retry.go (inline).

Out of scope, for the record: govulncheck ./... reports 9 stdlib + 4 imported-package vulnerabilities tree-wide; this PR changes no dependencies (go.mod/go.sum untouched, only stdlib imports added), so none are introduced here.

## Review: fix/retry-empty-model-response Retry-on-empty-output decorator for the ADK openaimodel client. Reviewed against a worktree at PR head `178fea0` (go1.27.1 toolchain, module go 1.27.1). ### Verification performed in this session - `go build ./...` → clean; `go vet ./internal/harness/` → clean; `go test -count=1 ./...` → all packages pass; `go test -race -count=1 ./internal/harness/` → ok (8.4s). - `golangci-lint run ./internal/harness/...` → only the pre-existing `varnamelen` on `NewRegistry`'s `f` parameter, exactly as the PR description states. - ADK claims confirmed against `google.golang.org/adk/v2@v2.5.0` module-cache source: `openaimodel.ErrNoOutputItems`/`ErrNoTextOrToolContent` exist (`model/openaimodel/errors.go:56,64`) as `errors.New` sentinels (`internal/shared/errors.go:54,67`), so the `errors.Is` matching in `emptyOutputError` is correct. `model.LLM` is `{ Name() string; GenerateContent(ctx, req, stream) iter.Seq2[*LLMResponse, error] }`; the decorator satisfies it. - Retry-trigger semantics match the real blocking path (`internal/responses/model.go:49-53`): a degenerate conversion yields exactly one `(nil, err)` pair, so the wrapper's "single pair is an empty-output error → retry" logic is exact. On both the Responses and Chat Completions paths, a tool call always appends a part before any empty-output error can fire (`convertOutputItems` / `convertMessage`), so the PR's safety claim "an empty response executed no tool calls" holds — the retry cannot double-run a tool. ### Findings No blocking or important issues. Three suggestions, all in `internal/harness/retry.go` (inline). Out of scope, for the record: `govulncheck ./...` reports 9 stdlib + 4 imported-package vulnerabilities tree-wide; this PR changes no dependencies (`go.mod`/`go.sum` untouched, only stdlib imports added), so none are introduced here.
@ -0,0 +17,4 @@
// emptyOutputError reports whether err means the provider answered with nothing
// usable (no text, no tool call).
func emptyOutputError(err error) bool {
Collaborator

🟡 SUGGESTION — emptyOutputError covers two of the three sentinels ADK itself treats as "empty output": shared.IsEmptyOutput (model/openaimodel/internal/shared/response.go:48) also includes ErrNoChoices. Brainless is Responses-API-only (openaimodel.NewModel defaults cfg.API == "" to APIResponses; nothing in internal/config sets the chat-completions API), so ErrNoChoices is unreachable today — but the divergence is a trap for the next editor. Either mirror IsEmptyOutput or comment why ErrNoChoices is deliberately excluded. Minor: the name reads as a noun; isEmptyOutputError would match Go predicate conventions.

🟡 SUGGESTION — `emptyOutputError` covers two of the three sentinels ADK itself treats as "empty output": `shared.IsEmptyOutput` (`model/openaimodel/internal/shared/response.go:48`) also includes `ErrNoChoices`. Brainless is Responses-API-only (`openaimodel.NewModel` defaults `cfg.API == ""` to `APIResponses`; nothing in `internal/config` sets the chat-completions API), so `ErrNoChoices` is unreachable today — but the divergence is a trap for the next editor. Either mirror `IsEmptyOutput` or comment why `ErrNoChoices` is deliberately excluded. Minor: the name reads as a noun; `isEmptyOutputError` would match Go predicate conventions.
@ -0,0 +49,4 @@
}
}
var head []emptyYield
Collaborator

🟡 SUGGESTION — emptyYield's resp field is always nil: pairs are buffered only in the resp == nil branch (line 67) and re-yielded as (nil, err) (lines 73, 87). A plain head []error would be equivalent and drops the dead field. Related ordering note: a (nil, err) pair arriving before a response in one sequence would be re-emitted after it (buffered, flushed post-delivered); no ADK path produces that shape today (error pairs are terminal — yield(nil, err); return in internal/responses/model.go), so this is theoretical only.

🟡 SUGGESTION — `emptyYield`'s `resp` field is always nil: pairs are buffered only in the `resp == nil` branch (line 67) and re-yielded as `(nil, err)` (lines 73, 87). A plain `head []error` would be equivalent and drops the dead field. Related ordering note: a `(nil, err)` pair arriving *before* a response in one sequence would be re-emitted *after* it (buffered, flushed post-`delivered`); no ADK path produces that shape today (error pairs are terminal — `yield(nil, err); return` in `internal/responses/model.go`), so this is theoretical only.
@ -0,0 +79,4 @@
}
// Retry only a degenerate empty output; surface anything else.
if len(head) > 0 && emptyOutputError(head[0].err) && attempt < maxEmptyOutputRetries {
Collaborator

🟡 SUGGESTION — retries fire silently. For a transient degenerate response (the reported case) that is fine, but if a provider degrades persistently, every turn pays up to 3× the price with no signal in logs — this wrapper sits below selectorModel, where the harness's slog wiring lives. Consider logging at debug/warn when a retry triggers (attempt number, r.llm.Name()), which also makes the retry cost path observable.

🟡 SUGGESTION — retries fire silently. For a transient degenerate response (the reported case) that is fine, but if a provider degrades persistently, every turn pays up to 3× the price with no signal in logs — this wrapper sits below `selectorModel`, where the harness's slog wiring lives. Consider logging at debug/warn when a retry triggers (attempt number, `r.llm.Name()`), which also makes the retry cost path observable.
- Mirror ADK's shared.IsEmptyOutput (add ErrNoChoices) so the predicate
  cannot diverge from upstream; rename emptyOutputError to
  isEmptyOutputError per Go predicate conventions.
- Drop the always-nil resp field of emptyYield; buffer errors only.
- Log a warning when a retry triggers (model, attempt, max attempts) so
  a persistently degrading provider's extra cost is observable.
Author
Collaborator

Review comments addressed in commit f619637:

  • #issuecomment-1031 — emptyOutputError renamed to isEmptyOutputError (Go predicate convention) and now mirrors ADK's shared.IsEmptyOutput exactly, including ErrNoChoices. shared is an internal ADK package brainless cannot import, so it is mirrored with a comment noting ErrNoChoices is Chat-Completions-only and currently unreachable.
  • #issuecomment-1032 — emptyYield.resp was always nil; the type is gone, head is now []error.
  • #issuecomment-1033 — retries now log a warning (component=retry) with model, attempt, and max_attempts, using the same slog.Default()-derived logger pattern as mcpLogger, so a persistently degrading provider's 3× cost is observable.

Verification: go build ./..., go vet ./internal/harness/, go test -count=1 ./..., and go test -race -count=1 ./internal/harness/ all pass; golangci-lint run ./internal/harness/ shows only the pre-existing varnamelen on NewRegistry. Note: the inline comments could not be marked resolved from the agent sandbox (no API access for the resolve endpoint); please resolve them in the UI.

Review comments addressed in commit f619637: - **#issuecomment-1031** — `emptyOutputError` renamed to `isEmptyOutputError` (Go predicate convention) and now mirrors ADK's `shared.IsEmptyOutput` exactly, including `ErrNoChoices`. `shared` is an internal ADK package brainless cannot import, so it is mirrored with a comment noting `ErrNoChoices` is Chat-Completions-only and currently unreachable. - **#issuecomment-1032** — `emptyYield.resp` was always nil; the type is gone, `head` is now `[]error`. - **#issuecomment-1033** — retries now log a warning (`component=retry`) with model, attempt, and max_attempts, using the same `slog.Default()`-derived logger pattern as `mcpLogger`, so a persistently degrading provider's 3× cost is observable. Verification: `go build ./...`, `go vet ./internal/harness/`, `go test -count=1 ./...`, and `go test -race -count=1 ./internal/harness/` all pass; `golangci-lint run ./internal/harness/` shows only the pre-existing `varnamelen` on `NewRegistry`. Note: the inline comments could not be marked resolved from the agent sandbox (no API access for the resolve endpoint); please resolve them in the UI.
sam merged commit c94d2adba2 into main 2026-10-12 07:14:22 +00:00
sam deleted branch fix/retry-empty-model-response 2026-10-12 07:14:22 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!29
No description provided.