A response containing only a thinking block silently ends the turn (the case #10353 didn't cover)
- Lingua principale
- Rust
- Stelle
- 54.2k
- Fork
- 6.2k
- Merge medio
- 3g 2h
- PR unite (30g)
- 262
Descrizione
**Describe the bug**
When a provider returns a response whose only content is a reasoning block — no text, no tool call —
the reply loop treats the turn as productive and exits. Nothing is rendered, nothing is logged, and
no retry is attempted. From the user's side the agent stops mid-task with no output and no error.
That is the same symptom as #10353 (*"Empty provider turn (no tool calls, no text) silently ends the
reply loop with no response or error"*), which was closed as completed by #10360. That fix covers a
response with **no content at all**; it does not cover a response with **reasoning and nothing else**,
because reasoning counts as content:
```rust
// crates/goose/src/agents/agent.rs:2800 (provider_produced_content)
MessageContent::Thinking(thinking) => {
!thinking.thinking.is_empty() || !thinking.signature.is_empty()
}
```
```rust
// crates/goose/src/agents/agent.rs:3437 (the guard)
let empty_response = no_tools_called
&& !exit_chat
&& !provider_errored
&& !did_recovery_compact_this_iteration
&& !provider_reached_output_token_limit
&& !provider_produced_content // <- true for a thinking-only response
&& last_assistant_text.is_empty();
```
So `empty_response` is false, the retry arm at `agent.rs:3528` is guarded `if empty_response` and is
skipped, and control reaches the silent fall-through:
```rust
// crates/goose/src/agents/agent.rs:3561
Ok(_) => {
exit_chat = true;
}
```
The state-machine loop has the same hole from the other direction — reasoning is only "empty" when
its text is blank, so a *non-empty* thinking-only response is likewise not empty there:
```rust
// crates/goose/src/agents/state_machine/ops_llm.rs:567
MessageContent::Thinking(thinking) => thinking.thinking.trim().is_empty(),
```
**To Reproduce**
Save as `repro.py`. Stdlib only — no gateway, proxy, credentials or real model involved.
```python
#!/usr/bin/env python3
"""Minimal OpenAI-compatible endpoint that returns an unproductive stream.
Usage: python3 repro.py [thinking-only|zero-content|control]
"""
import json
import sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
MODEL = "gpt-4o"
CASE = sys.argv[1] if len(sys.argv) > 1 else "thinking-only"
def chunk(delta, finish=None):
return {
"id": "chatcmpl-repro",
"object": "chat.completion.chunk",
"created": 1,
"model": MODEL,
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}],
}
def final():
done = chunk({}, "stop")
done["usage"] = {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
return done
FRAMES = {
# Reasoning arrives, then the stream stops. No content delta is ever sent.
"thinking-only": [
chunk({"role": "assistant", "reasoning_content": "Let me work through this."}),
chunk({"reasoning_content": " I have what I need now."}),
final(),
],
# Well-formed stream, correct finish_reason, no content at all.
"zero-content": [final()],
"control": [chunk({"role": "assistant", "content": "hello"}), final()],
}
class Handler(BaseHTTPRequestHandler):
def _send(self, body: bytes, ctype: str) -> None:
self.send_response(200)
self.send_header("Content-Type", ctype)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self): # /v1/models
body = json.dumps({"object": "list", "data": [{"id": MODEL, "object": "model"}]})
self._send(body.encode(), "application/json")
def do_POST(self): # /v1/chat/completions
self.rfile.read(int(self.headers.get("content-length") or 0))
body = "".join(f"data: {json.dumps(f)}\n\n" for f in FRAMES[CASE])
body += "data: [DONE]\n\n"
self._send(body.encode(), "text/event-stream")
print(f"listening on http://127.0.0.1:8111 (case={CASE})")
ThreadingHTTPServer(("127.0.0.1", 8111), Handler).serve_forever()
```
```bash
python3 repro.py thinking-only
# in another shell:
OPENAI_HOST=http://127.0.0.1:8111 OPENAI_API_KEY=dummy GOOSE_DISABLE_KEYRING=true \
goose run --provider openai --model gpt-4o --no-session -t "hi"
```
Run it with `zero-content` for the contrast: that case is detected and retried, and the CLI prints
`The model returned an empty response. Please resend your message to continue.` With
`thinking-only`, goose exits having printed nothing whatsoever.
**Expected behavior**
A turn that produced only reasoning has not answered the user. It should be retried like any other
unproductive turn, and if it persists, say so. It should never end the turn in silence.
**Actual behavior**
`goose run` exits `rc=0` in ~0.5 s having printed only the banner — no answer, no error. No warning
is logged either, so the failure is invisible to the user and to anyone reading logs afterwards.
Measured against a 1.48.0 build, driving the server above:
| `--case` | output |
|---|---|
| `control` | `hello` |
| `zero-content` | `The model returned an empty response. Please resend your message to continue.` |
| `thinking-only` | *(nothing at all)* |
`control` is the important row: it shows the fake endpoint, the `OPENAI_HOST` wiring and the SSE
parsing are all fine, so the empty output on `thinking-only` is the bug rather than a broken
harness. `zero-content` shows the handled case for contrast — and note it surfaces its message in
**0.2 s**, i.e. all four attempts complete before a user could notice, which is follow-up 1 below.
**Evidence from a real deployment**
One 98-turn session against Claude Opus behind an OpenAI-compatible gateway, with extended thinking
enabled. The gateway intermittently terminated streams early, producing both shapes side by side:
- **2 thinking-only turns.** A single `Thinking` block of 129 and 309 chars, `signature` absent, no
text, no tool call. Each ended the turn silently; the user waited **19.3 and 15.3 minutes** before
giving up and prodding manually. Both times the very next turn answered normally.
- **5 zero-content turns**, which *were* detected and retried — the log shows
`Provider returned an empty response; retrying (1/3)` four times per incident.
- The backend log contains **20 empty-response warnings** for the zero-content turns and **not one
line** for either thinking-only turn.
**Not a max_tokens problem** (re #11142, *"Reasoning models produce empty content: max_tokens shared
between reasoning_content and content"*): the two thinking-only turns emitted **43 and 478 output
tokens against a 128,000 `max_tokens`**, and `provider_reached_output_token_limit` was false. Nothing
was exhausted, so that issue's stated cause does not explain this shape — though the symptom is the
same and a fix here would likely cover it too.
**Suggested fix**
Track "did the provider produce an *answer*" separately from "did it produce content", and key the
empty-turn guard off the former:
```rust
provider_produced_answer |= response.content.iter().any(|content| match content {
MessageContent::Text(text) => !text.text.is_empty(),
MessageContent::Image(image) => !image.data.is_empty(),
MessageContent::Thinking(_) | MessageContent::RedactedThinking(_) => false,
MessageContent::SystemNotification(n) => !n.msg.is_empty(),
_ => true,
});
```
This is safe by construction: thinking followed by a tool call is already excluded by the existing
`no_tools_called` conjunct, and thinking followed by text has non-empty text. Only "thought, then
stopped" is newly caught. Worth giving it its own log line and its own user-facing wording, so the
transcript distinguishes "reasoned but never answered" from "returned nothing".
With that change plus follow-up 1 below, the same three cases behave like this — `thinking-only`
now retries on the backoff ladder and then says what happened, and with the server recovering after
two bad responses the turn completes normally:
```
$ goose run ... # --case thinking-only, served forever
· No answer from the model — retrying in 808ms (1/4)
· No answer from the model — retrying in 2s (2/4)
· No answer from the model — retrying in 4s (3/4)
· No answer from the model — retrying in 8s (4/4)
The model finished thinking but never produced an answer. Ask me to continue and I'll retry.
$ goose run ... # --case thinking-only, server recovers after 2 bad responses
· No answer from the model — retrying in 1s (1/4)
hello
```
---
**Two follow-ups on #10360's code, worth folding into the same fix**
**1. The empty-turn retries have no backoff.** `agent.rs` sets `retrying_after_empty_turn` and
re-enters the loop immediately. In the deployment above that meant 4 provider calls inside 6.3 s
(`18:58:11.94 → 14.03 → 16.29 → 18.28`), all unproductive — the entire retry budget spent inside the
few seconds the provider was unhealthy. A fresh turn 18–30 s later succeeded **every time, five times
out of five**. 20 provider calls bought zero recoveries.
`RetryConfig::delay_for_attempt` (`crates/goose-provider-types/src/retry.rs`) already implements
exponential backoff with ±20 % jitter and a cap, and is not otherwise reachable here because an
unproductive response is an HTTP 200 and never becomes a `ProviderError`. Reusing it, selecting on
the turn's `CancellationToken` so the user can still interrupt, is a small change.
**2. The fallback message is persisted agent-visible, so it is replayed to the model.**
```rust
// crates/goose/src/agents/agent.rs:3546
Message::assistant().with_text(EMPTY_TURN_MESSAGE), // defaults to user_visible + agent_visible
```
In a captured request payload from that session (53 messages), the string
`"The model returned an empty response. Please resend your message to continue."` appears as an
**assistant turn three separate times** — the model is repeatedly shown words it never produced.
This also contradicts the intent stated a few lines earlier at `agent.rs:3433`, where the empty
assistant turn is dropped precisely because *"strict providers reject a conversation that contains
an empty assistant turn."* `.with_visibility(true, false)` is the one-line fix; the API is already
used a few lines above.
**3. A regression worth checking separately.** `5370baae3` *"continue queued steer after empty
response"* added a `steering_queue.has_pending()` check to `ops_llm.rs` so a queued steer would
continue instead of yielding. That check appears to have been lost in the `goose-agent` extraction
refactor (`8c6106b1f` / `724250c42`), with no equivalent in `ops_steer.rs` today.
**Also worth noting:** the state-machine loop has no empty-turn retry at all — the first unproductive
response ends the turn. That gap matters before `GOOSE_STATE_MACHINE` becomes the default, since
switching over would otherwise regress exactly this case.
---
**Please provide the following information**
- **OS & Arch:** macOS 14 (Darwin 23.6.0), arm64
- **Interface:** CLI (reproduces in the desktop UI too)
- **Version:** 1.48.0
- **Extensions enabled:** none needed for the repro
- **Provider & Model:** `openai` provider against the local fake endpoint above. Originally hit on
Claude Opus behind an OpenAI-compatible gateway, with extended thinking enabled.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.