fix: retry degenerate model output until the provider answers or the budget runs out #30
No reviewers
Labels
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
sam/brainless!30
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/empty-output-retry-budget"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What breaks
Turns fail with
adk run: openai: response output did not contain text or tool content.The text is ADK's sentinel
openaimodel.ErrNoTextOrToolContent(google.golang.org/adk/v2/model/openaimodel): the provider answered with HTTP 200 and at least one output item, but nothing in that output converted to text or a tool call. The provider here is melious (https://api.melious.ai/v1), modeldeepseek-v4-flash-0731, and the degenerate answer is one assistant message whose only content is{"type":"output_text","text":""}with nofunction_call, i.e. a completed turn carrying nothing.retryEmptyOutputininternal/harness/retry.go(added in #29) re-sent such a response only twice. Measured against melious, a tool-bearing request comes back degenerate roughly two times in three, and every model round of a turn draws again, so a run of three degenerate answers — likely per round, near-certain per turn — still failed the turn.Evidence
https://api.melious.ai/v1captured the exact request bodies brainless sends and the provider's answers. Replaying one captured body 12 times returned 10 degenerate answers and 2 usable ones, so the flakiness is per request, not a fixed property of that prompt./v1/responsesand 8 times out of 8 on/v1/chat/completions, while the request carrying brainless's 10 tool schemas plus its instructions was degenerate 10 times out of 12.Note the provider reports
output_tokens: 1andoutput_tokens_details.reasoning_tokens: 1on every answer, including usable ones, so its usage block cannot be used to detect the degenerate case.Change
retryEmptyOutputnow re-sends a degenerate answer while both limits allow:maxEmptyOutputRetries = 30(was 2) — a cap, so a provider that answers degenerately in microseconds cannot turn the time budget into thousands of billed requests.emptyOutputRetryBudget = 15 * time.Second(new) — the wall-clock bound, so a provider that never produces a usable answer still fails the turn in bounded time instead of retrying forever.emptyOutputRetryDelaystays 200ms: degenerate answers arrive in a few hundred milliseconds and are billed at one output token, so recovering a turn costs roughly a second.New test
TestRetryEmptyOutputAbsorbsLongDegenerateRuncovers a six-degenerate run, which the previous two retries could not absorb. The retry still never fires after a response has been delivered, so no tool call can be executed twice (unchanged from #29).go vet ./...andgo test ./...pass;golangci-lint run ./internal/harness/...reports only the pre-existingvarnamelenfinding onNewRegistry.Deploy note
The
paseocontainer downloads this binary from the rollinglatestrelease in thebrainless-fetchstage ofsam/pi-deploy'spaseo/Dockerfile. Docker caches that stage, so the container keeps serving an old binary unless the build forces it (docker buildx build --no-cache-filter brainless-fetch -t paseo:0.11.2 ./paseo, thendocker compose up -d --force-recreate paseo) and the result is checked (docker exec paseo stat -c '%y' /usr/local/bin/brainlessmust show the build date).