agentscope-ai / agentscope-ai/agentscope-java

[Feature]: Default to including usage field in OpenAI API completions/chat responses

未關閉
#355 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
area/core/model enhancement
主要語言
Java
星號
5.6k
分支
1.3k
平均合併
4 天 12 小時
30 天內合併 PR
77

描述

**Is your feature request related to a problem? Please describe.**
I'm always frustrated when using OpenAI-compatible APIs (e.g., Bailian from Alibaba Cloud) in streaming mode, because the final response chunk containing token usage information is not returned by default. AgentScope expects usage data (e.g., prompt_tokens, completion_tokens) in the last response for cost tracking and logging, but certain providers like Bailian require an explicit `stream_options: {"include_usage": true}` parameter to include this information in the stream.

**Describe the solution you'd like**
AgentScope should automatically include `stream_options: {"include_usage": true}` in the request payload when making streaming calls to OpenAI-compatible models, if the underlying provider supports it. This could be implemented as a built-in default or auto-enabled when usage tracking is requested, ensuring consistent behavior across different backends without requiring users to manually configure this option.

**Describe alternatives you've considered**
- Users could manually add `stream_options` to their model configuration, but this breaks the abstraction and requires provider-specific knowledge.
- AgentScope could detect the provider (e.g., via endpoint URL or model name) and conditionally inject the option, but this adds complexity.
- Alternatively, make it a configurable flag in the model wrapper (e.g., `include_usage_in_stream: true`), but that still shifts responsibility to the user.

**Additional context**
This issue commonly occurs with Alibaba Cloud’s Bailian API and potentially other OpenAI-compatible services that follow the newer OpenAI API spec where `include_usage` is opt-in for streaming responses. Without this option, the final message with `"usage"` is omitted, causing AgentScope to miss critical token usage metrics. Making this behavior default (or smartly inferred) would improve compatibility and user experience.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。