aaif-goose / aaif-goose/goose

Bedrock Converse ignores stopReason: max_tokens truncation is never reported and a cut-off tool call is dropped

Open
#11,838 1 comment 0 reactions 3 assignees Claimed by @lifeizhou-ap View on GitHub
Dominant language
Rust
Stars
54.2k
Forks
6.2k
Avg merge
3d 2h
Merged PRs (30d)
262

Description

**Describe the bug**

The Bedrock provider never reads `stopReason` on the Converse path, so the output-token-limit handling from #10831 has no effect for Claude on Bedrock. When a response is cut off by `maxTokens`:

- the CLI/Desktop truncation warning never renders. `crates/goose-cli/src/session/output.rs:463` checks `output_token_limit_reached`, and nothing in `bedrock.rs` ever sets it
- ACP `session/prompt` reports `end_turn` for the truncated turn (`crates/goose/src/acp/server.rs:742-746`)
- a tool call cut off mid-arguments is dropped silently. Its `ContentBlockStop` never arrives and `state.tool_blocks` is not flushed when the stream ends (`bedrock.rs:1058-1070`), so the agent sees a turn with no tool request and finishes as if the model was done
- `usage.finish_reasons` stays empty, so `gen_ai.response.finish_reasons` is missing from Bedrock spans (`crates/goose/src/agents/gen_ai_telemetry.rs:70-72`)

Where it happens: streaming `MessageStop` lands in the catch-all arm at `crates/goose/src/providers/bedrock.rs:834-836` ("MessageStart / MessageStop / unknown variants carry no content that needs forwarding"). Non-streaming (`BEDROCK_DISABLE_STREAMING=true`) returns `(response.output, response.usage)` and drops `ConverseOutput.stop_reason` at `bedrock.rs:562-567`. The field is required in both shapes: [MessageStopEvent](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_MessageStopEvent.html) lists `stopReason` as Required with `max_tokens` among the valid values, and in `aws-sdk-bedrockruntime` 1.135.0 it is a non-optional `StopReason` with a `MaxTokens` variant.

Who it affects: every model that goes through Converse. That is the `anthropic.claude-*` entries in `BEDROCK_MODEL_TABLE` plus any model id not in the table at all (`find_model_entry` returns `None` and `stream()` falls through to `converse_stream`), which is what you get with a custom inference profile id. The Mantle entries (`openai.gpt-5.*`, `google.gemma-4-*`) are not affected; they go through `stream_responses_compat` and the Responses formatter already reads `incomplete_details` (`openai_responses.rs:1128`).

Split out from #10950, which jbg scoped to native Google. I wrote up the Bedrock gap in a [comment there](https://github.com/aaif-goose/goose/issues/10950#issuecomment-5459466323) with the tests and a sketch of the streaming arm.

---

**To Reproduce**

No credentials needed. On main (c59e45d), drive `process_stream_event` directly:

1. `MessageStop(max_tokens)` on its own. Expected: a message with `output_token_limit_reached = true`. Actual: `[]`.
2. `ContentBlockStart(toolUse)`, one partial `ContentBlockDelta` (`{"path":"/report.md",`), then `MessageStop(max_tokens)` with no `ContentBlockStop`. Expected: a failed tool request. Actual: nothing yielded.
3. Control: `MessageStop(end_turn)` must not set the flag.

```
$ cargo test -p goose --features aws-providers --lib providers::bedrock::tests::test_stream

test providers::bedrock::tests::test_stream_message_stop_max_tokens_marks_output_token_limit ... FAILED
test providers::bedrock::tests::test_stream_tool_call_cut_off_before_content_block_stop ... FAILED

---- providers::bedrock::tests::test_stream_message_stop_max_tokens_marks_output_token_limit stdout ----
panicked at crates/goose/src/providers/bedrock.rs:1993:9:
max_tokens stop should mark the turn output-token-limited, yielded: []

---- providers::bedrock::tests::test_stream_tool_call_cut_off_before_content_block_stop stdout ----
panicked at crates/goose/src/providers/bedrock.rs:2039:9:
a tool call cut off by the output limit should surface to the agent, yielded nothing

test result: FAILED. 12 passed; 2 failed; 0 ignored
```

The three tests are in the #10950 comment linked above.

---

**Expected behavior**

Same as Claude through the Anthropic provider after #10831: partial text kept, `output_token_limit_reached` set on the turn so the CLI/Desktop warning renders and ACP returns `max_tokens`, a tool call left open at the cut-off surfaced as a failed tool request instead of vanishing (the Anthropic formatter does this at `crates/goose-provider-types/src/formats/anthropic.rs:1199-1233`), and `finish_reasons` populated.

---

**Please provide the following information**
- **OS & Arch:** macOS arm64 (code-path bug, not OS specific)
- **Interface:** CLI, Desktop and ACP all consume the same flag
- **Version:** main `c59e45d` (v1.49.0)
- **Extensions enabled:** n/a
- **Provider & Model:** Amazon Bedrock – `us.anthropic.claude-sonnet-4-5-20250929-v1:0` (Converse endpoint)

---

**Additional context**

Direction, for the design discussion: on the streaming side, one `MessageStop` arm in `process_stream_event` guarded on `MaxTokens` that does what the Anthropic formatter does at stream end. Flush any still-open tool block as a failed tool request, then push the metadata-only marker. `truncation_error_message` in `crates/goose-provider-types/src/json.rs:174` already builds that error text for the OpenAI formatters, so no new string or mechanism. I have this passing locally: the two tests above go green, `providers::bedrock` is 31/31, fmt and clippy are clean.

The non-streaming path is a different size of change. `converse()` returns `(Message, Option)` and would need to hand the stop reason back to `stream_via_converse`. Worth deciding here whether the escape hatch is in scope or a follow-up.

One thing I could not settle without a live capture: whether Bedrock sends `contentBlockStop` for the open block before `messageStop` on a `maxTokens` cut-off. If it does, the existing invalid-JSON arm already yields an error request and only the marker is missing. Either way the marker is missing. The verification plan should include one live run with a small `GOOSE_MAX_TOKENS` against Claude on Bedrock to confirm the warning renders and ACP reports `max_tokens`. Happy to pick this up once it reaches Ready.

Related: #10831, #10950

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.