fix: retry degenerate model output until the provider answers or the budget runs out #30

Merged
sam merged 2 commits from fix/empty-output-retry-budget into main 2026-10-12 20:37:02 +00:00
Collaborator

What breaks

Turns fail with adk run: openai: response output did not contain text or tool content.

The text is ADK's sentinel openaimodel.ErrNoTextOrToolContent (google.golang.org/adk/v2/model/openaimodel): the provider answered with HTTP 200 and at least one output item, but nothing in that output converted to text or a tool call. The provider here is melious (https://api.melious.ai/v1), model deepseek-v4-flash-0731, and the degenerate answer is one assistant message whose only content is {"type":"output_text","text":""} with no function_call, i.e. a completed turn carrying nothing.

retryEmptyOutput in internal/harness/retry.go (added in #29) re-sent such a response only twice. Measured against melious, a tool-bearing request comes back degenerate roughly two times in three, and every model round of a turn draws again, so a run of three degenerate answers — likely per round, near-certain per turn — still failed the turn.

Evidence

  • A logging reverse proxy in front of https://api.melious.ai/v1 captured the exact request bodies brainless sends and the provider's answers. Replaying one captured body 12 times returned 10 degenerate answers and 2 usable ones, so the flakiness is per request, not a fixed property of that prompt.
  • The same traffic also shows the trigger: a minimal request with no tool schemas answered 8 times out of 8 on /v1/responses and 8 times out of 8 on /v1/chat/completions, while the request carrying brainless's 10 tool schemas plus its instructions was degenerate 10 times out of 12.
  • Offline, against a stub provider that returns six degenerate answers and then a usable one, the previous code failed after a single request; with this change the same run makes seven requests and returns the answer in about a second.

Note the provider reports output_tokens: 1 and output_tokens_details.reasoning_tokens: 1 on every answer, including usable ones, so its usage block cannot be used to detect the degenerate case.

Change

retryEmptyOutput now re-sends a degenerate answer while both limits allow:

  • maxEmptyOutputRetries = 30 (was 2) — a cap, so a provider that answers degenerately in microseconds cannot turn the time budget into thousands of billed requests.
  • emptyOutputRetryBudget = 15 * time.Second (new) — the wall-clock bound, so a provider that never produces a usable answer still fails the turn in bounded time instead of retrying forever.
  • emptyOutputRetryDelay stays 200ms: degenerate answers arrive in a few hundred milliseconds and are billed at one output token, so recovering a turn costs roughly a second.

New test TestRetryEmptyOutputAbsorbsLongDegenerateRun covers a six-degenerate run, which the previous two retries could not absorb. The retry still never fires after a response has been delivered, so no tool call can be executed twice (unchanged from #29).

go vet ./... and go test ./... pass; golangci-lint run ./internal/harness/... reports only the pre-existing varnamelen finding on NewRegistry.

Deploy note

The paseo container downloads this binary from the rolling latest release in the brainless-fetch stage of sam/pi-deploy's paseo/Dockerfile. Docker caches that stage, so the container keeps serving an old binary unless the build forces it (docker buildx build --no-cache-filter brainless-fetch -t paseo:0.11.2 ./paseo, then docker compose up -d --force-recreate paseo) and the result is checked (docker exec paseo stat -c '%y' /usr/local/bin/brainless must show the build date).

## What breaks Turns fail with `adk run: openai: response output did not contain text or tool content`. The text is ADK's sentinel `openaimodel.ErrNoTextOrToolContent` (`google.golang.org/adk/v2/model/openaimodel`): the provider answered with HTTP 200 and at least one output item, but nothing in that output converted to text or a tool call. The provider here is melious (`https://api.melious.ai/v1`), model `deepseek-v4-flash-0731`, and the degenerate answer is one assistant message whose only content is `{"type":"output_text","text":""}` with no `function_call`, i.e. a completed turn carrying nothing. `retryEmptyOutput` in `internal/harness/retry.go` (added in #29) re-sent such a response only twice. Measured against melious, a tool-bearing request comes back degenerate roughly two times in three, and every model round of a turn draws again, so a run of three degenerate answers — likely per round, near-certain per turn — still failed the turn. ## Evidence - A logging reverse proxy in front of `https://api.melious.ai/v1` captured the exact request bodies brainless sends and the provider's answers. Replaying one captured body 12 times returned 10 degenerate answers and 2 usable ones, so the flakiness is per request, not a fixed property of that prompt. - The same traffic also shows the trigger: a minimal request with no tool schemas answered 8 times out of 8 on `/v1/responses` and 8 times out of 8 on `/v1/chat/completions`, while the request carrying brainless's 10 tool schemas plus its instructions was degenerate 10 times out of 12. - Offline, against a stub provider that returns six degenerate answers and then a usable one, the previous code failed after a single request; with this change the same run makes seven requests and returns the answer in about a second. Note the provider reports `output_tokens: 1` and `output_tokens_details.reasoning_tokens: 1` on every answer, including usable ones, so its usage block cannot be used to detect the degenerate case. ## Change `retryEmptyOutput` now re-sends a degenerate answer while both limits allow: - `maxEmptyOutputRetries = 30` (was 2) — a cap, so a provider that answers degenerately in microseconds cannot turn the time budget into thousands of billed requests. - `emptyOutputRetryBudget = 15 * time.Second` (new) — the wall-clock bound, so a provider that never produces a usable answer still fails the turn in bounded time instead of retrying forever. - `emptyOutputRetryDelay` stays 200ms: degenerate answers arrive in a few hundred milliseconds and are billed at one output token, so recovering a turn costs roughly a second. New test `TestRetryEmptyOutputAbsorbsLongDegenerateRun` covers a six-degenerate run, which the previous two retries could not absorb. The retry still never fires after a response has been delivered, so no tool call can be executed twice (unchanged from #29). `go vet ./...` and `go test ./...` pass; `golangci-lint run ./internal/harness/...` reports only the pre-existing `varnamelen` finding on `NewRegistry`. ## Deploy note The `paseo` container downloads this binary from the rolling `latest` release in the `brainless-fetch` stage of `sam/pi-deploy`'s `paseo/Dockerfile`. Docker caches that stage, so the container keeps serving an old binary unless the build forces it (`docker buildx build --no-cache-filter brainless-fetch -t paseo:0.11.2 ./paseo`, then `docker compose up -d --force-recreate paseo`) and the result is checked (`docker exec paseo stat -c '%y' /usr/local/bin/brainless` must show the build date).
The retry added for "openai: response output did not contain text or tool
content" gave up after two re-sends. Measured against the provider that
produces the error (melious, model deepseek-v4-flash-0731, endpoint
https://api.melious.ai/v1/responses), a request carrying the harness's tool
schemas comes back as an empty turn about two times in three: HTTP 200 with
one assistant message whose output_text is "" and no function call, which
ADK's Responses converter turns into openaimodel.ErrNoTextOrToolContent. A
replayed identical request body produced 10 empty answers out of 12, while a
minimal request (no tools) answered 8 times out of 8, so the emptiness is the
provider's, not the request's. With that rate and one draw per model round,
each turn of a conversation draws several times, so two retries left whole
turns failing.

Retry now runs while the attempt cap (30) and a 15s wall-clock budget allow
whichever comes first. Degenerate answers arrive in a few hundred
milliseconds, so recovering a turn costs about a second, and a provider that
never answers keeps the failure bounded. The budget also keeps a provider
that answers degenerately in microseconds from spinning the cap into
thousands of billed requests.

Verified end to end against a local stub provider that returns six
degenerate responses and then a usable one: the previous code failed after a
single request, the new code makes seven requests and returns the answer.
sam merged commit 80e25415ec into main 2026-10-12 20:37:02 +00:00
sam deleted branch fix/empty-output-retry-budget 2026-10-12 20:37:03 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!30
No description provided.