agentscope-ai / agentscope-ai/agentscope-java
[Bug]:Anthropic streaming responses drop ChatUsage
- Lingua principale
- Java
- Stelle
- 5.6k
- Fork
- 1.3k
- Merge medio
- 4g 12h
- PR unite (30g)
- 77
Descrizione
**Describe the bug**
When `AnthropicChatModel` is used with streaming enabled, every
`ChatResponse` delivered to the caller has `usage == null`, even though
the Anthropic SSE response contains token usage information.
Non-streaming requests are not affected.
The Anthropic streaming protocol reports usage in separate events:
- `message_start`: contains `message.usage.input_tokens`
- `content_block_delta`: contains generated content but no usage
- `message_delta`: contains `usage.output_tokens`
Currently, the events containing usage are discarded by
`AnthropicResponseParser`.
**To Reproduce**
Create an Anthropic model with streaming enabled:
```java
Model model = AnthropicChatModel.builder()
.apiKey(apiKey)
.baseUrl(baseUrl)
.modelName(modelName)
.stream(true)
.build();
model.stream(messages, List.of(), GenerateOptions.builder().build())
.doOnNext(response ->
System.out.println("usage = " + response.getUsage()))
.blockLast();
```
Use an Anthropic endpoint that returns streaming usage events such as:
```
event: message_start
data: {
"type": "message_start",
"message": {
"usage": {
"input_tokens": 5,
"output_tokens": 0
}
}
}
event: content_block_delta
data: {
"type": "content_block_delta",
"delta": {
"type": "text_delta",
"text": "Hello"
}
}
event: message_delta
data: {
"type": "message_delta",
"usage": {
"output_tokens": 10
}
}
```
All emitted content responses print:
```
usage = null
```
**Expected behavior**
The stream should expose token usage to the caller.
At minimum, the final message_delta event should produce a
ChatResponse containing:
```
inputTokens = 5
outputTokens = 10
```
A usage-only ChatResponse with empty content would also be acceptable,
as long as it is not filtered out.
**Error messages**
No exception is thrown. The generated content is returned normally, but
ChatResponse.getUsage() is always null.
**Environment (please complete the following information):**
AgentScope-Java Version: 2.0.0-RC4
Also observed in: 2.0.0-RC5
Java Version: 21
OS: macOS
Model API: Anthropic Messages API / Anthropic-compatible gateway
Streaming: enabled
**Additional context**
The issue appears to be in AnthropicResponseParser.
parseStreamEvent() creates a new local ChatUsage for every event:
```java
ChatUsage usage = null;
```
For message_start, it reads only the message ID and does not read
message.usage.input_tokens.
For message_delta, it creates a ChatUsage containing only
outputTokens. However, this event has no content blocks.
parseStreamEvents() then filters responses using:
```java
.filter(response ->
response != null && !response.getContent().isEmpty());
```
Therefore:
message_start containing input usage is discarded because content is empty.
Content delta events are emitted, but their usage is null.
message_delta containing output usage is discarded because content is empty.
A possible fix would be to:
1. Keep usage state per stream subscription.
2. Read input_tokens from message_start.
3. Update output_tokens when message_delta arrives.
4. Preserve responses that contain usage even when content is empty:
```java
.filter(response ->
response != null
&& (!response.getContent().isEmpty()
|| response.getUsage() != null));
```
Thank you for taking a look.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.