agentscope-ai / agentscope-ai/agentscope-java

[Feature]: Default to including usage field in OpenAI API completions/chat responses

Aperta
#355 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
area/core/model enhancement
Lingua principale
Java
Stelle
5.6k
Fork
1.3k
Merge medio
4g 12h
PR unite (30g)
77

Descrizione

**Is your feature request related to a problem? Please describe.**
I'm always frustrated when using OpenAI-compatible APIs (e.g., Bailian from Alibaba Cloud) in streaming mode, because the final response chunk containing token usage information is not returned by default. AgentScope expects usage data (e.g., prompt_tokens, completion_tokens) in the last response for cost tracking and logging, but certain providers like Bailian require an explicit `stream_options: {"include_usage": true}` parameter to include this information in the stream.

**Describe the solution you'd like**
AgentScope should automatically include `stream_options: {"include_usage": true}` in the request payload when making streaming calls to OpenAI-compatible models, if the underlying provider supports it. This could be implemented as a built-in default or auto-enabled when usage tracking is requested, ensuring consistent behavior across different backends without requiring users to manually configure this option.

**Describe alternatives you've considered**
- Users could manually add `stream_options` to their model configuration, but this breaks the abstraction and requires provider-specific knowledge.
- AgentScope could detect the provider (e.g., via endpoint URL or model name) and conditionally inject the option, but this adds complexity.
- Alternatively, make it a configurable flag in the model wrapper (e.g., `include_usage_in_stream: true`), but that still shifts responsibility to the user.

**Additional context**
This issue commonly occurs with Alibaba Cloud’s Bailian API and potentially other OpenAI-compatible services that follow the newer OpenAI API spec where `include_usage` is opt-in for streaming responses. Without this option, the final message with `"usage"` is omitted, causing AgentScope to miss critical token usage metrics. Making this behavior default (or smartly inferred) would improve compatibility and user experience.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.