fix: retry model responses that carry no text or tool content #29
No reviewers
Labels
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
sam/brainless!29
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/retry-empty-model-response"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
Brainless turns in Paseo fail intermittently with:
Root cause: the Google ADK v2.5.0 blocking Responses path
(
model/openaimodel/internal/responses/response.go,convertOutputItems) failsthe whole turn when a provider response has output items but none of them
convert to a text or tool-call part — all
reasoning/messageitems arrivewith empty text. Melious intermittently returns exactly that shape for the
reasoning models brainless runs (seen on
melious/deepseek-v4-flash-0731inthe Oct 11 failure; sibling pi agent on the same model ran fine, and ~35
direct API calls reproduced no degenerate shape — a transient provider
response). ADK's streaming path tolerates empty outputs; brainless runs
StreamingModeNone, so the blocking path fails hard.Fix
Wrap every registry model client (
NewRegistry) in aretryEmptyOutputdecorator (
internal/harness/retry.go):only thing yielded is
openaimodel.ErrNoOutputItems/openaimodel.ErrNoTextOrToolContent.cannot double-run anything.
error after yielded responses is preserved.
summarization gets the same protection.
Tests:
internal/harness/retry_test.go(retry-once, exhaust-retries,no-retry-on-other-errors, preserve-trailing-error).
go test ./...,go test -race ./internal/harness/, andgolangci-lint run ./...all pass(the only lint finding is a pre-existing
varnamelenon the untouchedNewRegistrysignature).Note on streaming: switching the runner to streaming would hide this bug
(ADK streams degenerate responses as an empty successful turn), not fix it —
the user would get no answer and no error. Retry actively recovers; streaming
is a separate UX change.
60a00f6a23to178fea0e25Review: fix/retry-empty-model-response
Retry-on-empty-output decorator for the ADK openaimodel client. Reviewed against a worktree at PR head
178fea0(go1.27.1 toolchain, module go 1.27.1).Verification performed in this session
go build ./...→ clean;go vet ./internal/harness/→ clean;go test -count=1 ./...→ all packages pass;go test -race -count=1 ./internal/harness/→ ok (8.4s).golangci-lint run ./internal/harness/...→ only the pre-existingvarnamelenonNewRegistry'sfparameter, exactly as the PR description states.google.golang.org/adk/v2@v2.5.0module-cache source:openaimodel.ErrNoOutputItems/ErrNoTextOrToolContentexist (model/openaimodel/errors.go:56,64) aserrors.Newsentinels (internal/shared/errors.go:54,67), so theerrors.Ismatching inemptyOutputErroris correct.model.LLMis{ Name() string; GenerateContent(ctx, req, stream) iter.Seq2[*LLMResponse, error] }; the decorator satisfies it.internal/responses/model.go:49-53): a degenerate conversion yields exactly one(nil, err)pair, so the wrapper's "single pair is an empty-output error → retry" logic is exact. On both the Responses and Chat Completions paths, a tool call always appends a part before any empty-output error can fire (convertOutputItems/convertMessage), so the PR's safety claim "an empty response executed no tool calls" holds — the retry cannot double-run a tool.Findings
No blocking or important issues. Three suggestions, all in
internal/harness/retry.go(inline).Out of scope, for the record:
govulncheck ./...reports 9 stdlib + 4 imported-package vulnerabilities tree-wide; this PR changes no dependencies (go.mod/go.sumuntouched, only stdlib imports added), so none are introduced here.@ -0,0 +17,4 @@// emptyOutputError reports whether err means the provider answered with nothing// usable (no text, no tool call).func emptyOutputError(err error) bool {🟡 SUGGESTION —
emptyOutputErrorcovers two of the three sentinels ADK itself treats as "empty output":shared.IsEmptyOutput(model/openaimodel/internal/shared/response.go:48) also includesErrNoChoices. Brainless is Responses-API-only (openaimodel.NewModeldefaultscfg.API == ""toAPIResponses; nothing ininternal/configsets the chat-completions API), soErrNoChoicesis unreachable today — but the divergence is a trap for the next editor. Either mirrorIsEmptyOutputor comment whyErrNoChoicesis deliberately excluded. Minor: the name reads as a noun;isEmptyOutputErrorwould match Go predicate conventions.@ -0,0 +49,4 @@}}var head []emptyYield🟡 SUGGESTION —
emptyYield'srespfield is always nil: pairs are buffered only in theresp == nilbranch (line 67) and re-yielded as(nil, err)(lines 73, 87). A plainhead []errorwould be equivalent and drops the dead field. Related ordering note: a(nil, err)pair arriving before a response in one sequence would be re-emitted after it (buffered, flushed post-delivered); no ADK path produces that shape today (error pairs are terminal —yield(nil, err); returnininternal/responses/model.go), so this is theoretical only.@ -0,0 +79,4 @@}// Retry only a degenerate empty output; surface anything else.if len(head) > 0 && emptyOutputError(head[0].err) && attempt < maxEmptyOutputRetries {🟡 SUGGESTION — retries fire silently. For a transient degenerate response (the reported case) that is fine, but if a provider degrades persistently, every turn pays up to 3× the price with no signal in logs — this wrapper sits below
selectorModel, where the harness's slog wiring lives. Consider logging at debug/warn when a retry triggers (attempt number,r.llm.Name()), which also makes the retry cost path observable.Review comments addressed in commit
f619637:emptyOutputErrorrenamed toisEmptyOutputError(Go predicate convention) and now mirrors ADK'sshared.IsEmptyOutputexactly, includingErrNoChoices.sharedis an internal ADK package brainless cannot import, so it is mirrored with a comment notingErrNoChoicesis Chat-Completions-only and currently unreachable.emptyYield.respwas always nil; the type is gone,headis now[]error.component=retry) with model, attempt, and max_attempts, using the sameslog.Default()-derived logger pattern asmcpLogger, so a persistently degrading provider's 3× cost is observable.Verification:
go build ./...,go vet ./internal/harness/,go test -count=1 ./..., andgo test -race -count=1 ./internal/harness/all pass;golangci-lint run ./internal/harness/shows only the pre-existingvarnamelenonNewRegistry. Note: the inline comments could not be marked resolved from the agent sandbox (no API access for the resolve endpoint); please resolve them in the UI.