agentscope-ai / agentscope-ai/agentscope-java

[Feature]: Default to including usage field in OpenAI API completions/chat responses

オープン
#355 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
area/core/model enhancement
主要言語
Java
スター
5.6k
フォーク
1.3k
平均マージ
4日 12時間
マージ済み PR(30日)
77

説明

**Is your feature request related to a problem? Please describe.**
I'm always frustrated when using OpenAI-compatible APIs (e.g., Bailian from Alibaba Cloud) in streaming mode, because the final response chunk containing token usage information is not returned by default. AgentScope expects usage data (e.g., prompt_tokens, completion_tokens) in the last response for cost tracking and logging, but certain providers like Bailian require an explicit `stream_options: {"include_usage": true}` parameter to include this information in the stream.

**Describe the solution you'd like**
AgentScope should automatically include `stream_options: {"include_usage": true}` in the request payload when making streaming calls to OpenAI-compatible models, if the underlying provider supports it. This could be implemented as a built-in default or auto-enabled when usage tracking is requested, ensuring consistent behavior across different backends without requiring users to manually configure this option.

**Describe alternatives you've considered**
- Users could manually add `stream_options` to their model configuration, but this breaks the abstraction and requires provider-specific knowledge.
- AgentScope could detect the provider (e.g., via endpoint URL or model name) and conditionally inject the option, but this adds complexity.
- Alternatively, make it a configurable flag in the model wrapper (e.g., `include_usage_in_stream: true`), but that still shifts responsibility to the user.

**Additional context**
This issue commonly occurs with Alibaba Cloud’s Bailian API and potentially other OpenAI-compatible services that follow the newer OpenAI API spec where `include_usage` is opt-in for streaming responses. Without this option, the final message with `"usage"` is omitted, causing AgentScope to miss critical token usage metrics. Making this behavior default (or smartly inferred) would improve compatibility and user experience.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。