agentscope-ai / agentscope-ai/agentscope-java

[Bug]:Anthropic streaming responses drop ChatUsage

未关闭
#2,094 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
area/core/model bug
主要语言
Java
星标
5.6k
派生
1.3k
平均合并
4 天 12 小时
30 天内合并 PR
77

描述

**Describe the bug**
When `AnthropicChatModel` is used with streaming enabled, every
`ChatResponse` delivered to the caller has `usage == null`, even though
the Anthropic SSE response contains token usage information.

Non-streaming requests are not affected.

The Anthropic streaming protocol reports usage in separate events:

- `message_start`: contains `message.usage.input_tokens`
- `content_block_delta`: contains generated content but no usage
- `message_delta`: contains `usage.output_tokens`

Currently, the events containing usage are discarded by
`AnthropicResponseParser`.

**To Reproduce**
Create an Anthropic model with streaming enabled:

```java
Model model = AnthropicChatModel.builder()
.apiKey(apiKey)
.baseUrl(baseUrl)
.modelName(modelName)
.stream(true)
.build();

model.stream(messages, List.of(), GenerateOptions.builder().build())
.doOnNext(response ->
System.out.println("usage = " + response.getUsage()))
.blockLast();
```
Use an Anthropic endpoint that returns streaming usage events such as:

```
event: message_start
data: {
"type": "message_start",
"message": {
"usage": {
"input_tokens": 5,
"output_tokens": 0
}
}
}

event: content_block_delta
data: {
"type": "content_block_delta",
"delta": {
"type": "text_delta",
"text": "Hello"
}
}

event: message_delta
data: {
"type": "message_delta",
"usage": {
"output_tokens": 10
}
}
```
All emitted content responses print:

```
usage = null
```

**Expected behavior**
The stream should expose token usage to the caller.
At minimum, the final message_delta event should produce a
ChatResponse containing:

```
inputTokens = 5
outputTokens = 10
```
A usage-only ChatResponse with empty content would also be acceptable,
as long as it is not filtered out.

**Error messages**
No exception is thrown. The generated content is returned normally, but
ChatResponse.getUsage() is always null.

**Environment (please complete the following information):**

AgentScope-Java Version: 2.0.0-RC4
Also observed in: 2.0.0-RC5
Java Version: 21
OS: macOS
Model API: Anthropic Messages API / Anthropic-compatible gateway
Streaming: enabled

**Additional context**
The issue appears to be in AnthropicResponseParser.
parseStreamEvent() creates a new local ChatUsage for every event:

```java
ChatUsage usage = null;
```

For message_start, it reads only the message ID and does not read
message.usage.input_tokens.
For message_delta, it creates a ChatUsage containing only
outputTokens. However, this event has no content blocks.
parseStreamEvents() then filters responses using:

```java
.filter(response ->
response != null && !response.getContent().isEmpty());
```

Therefore:
message_start containing input usage is discarded because content is empty.
Content delta events are emitted, but their usage is null.
message_delta containing output usage is discarded because content is empty.
A possible fix would be to:
1. Keep usage state per stream subscription.
2. Read input_tokens from message_start.
3. Update output_tokens when message_delta arrives.
4. Preserve responses that contain usage even when content is empty:

```java
.filter(response ->
response != null
&& (!response.getContent().isEmpty()
|| response.getUsage() != null));
```
Thank you for taking a look.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。