agentscope-ai / agentscope-ai/agentscope-java

[Feature]: Default to including usage field in OpenAI API completions/chat responses

Open
#355 1 comment 0 reactions 0 assignees View on GitHub
area/core/model enhancement
Dominant language
Java
Stars
5.6k
Forks
1.3k
Avg merge
4d 12h
Merged PRs (30d)
77

Description

**Is your feature request related to a problem? Please describe.**
I'm always frustrated when using OpenAI-compatible APIs (e.g., Bailian from Alibaba Cloud) in streaming mode, because the final response chunk containing token usage information is not returned by default. AgentScope expects usage data (e.g., prompt_tokens, completion_tokens) in the last response for cost tracking and logging, but certain providers like Bailian require an explicit `stream_options: {"include_usage": true}` parameter to include this information in the stream.

**Describe the solution you'd like**
AgentScope should automatically include `stream_options: {"include_usage": true}` in the request payload when making streaming calls to OpenAI-compatible models, if the underlying provider supports it. This could be implemented as a built-in default or auto-enabled when usage tracking is requested, ensuring consistent behavior across different backends without requiring users to manually configure this option.

**Describe alternatives you've considered**
- Users could manually add `stream_options` to their model configuration, but this breaks the abstraction and requires provider-specific knowledge.
- AgentScope could detect the provider (e.g., via endpoint URL or model name) and conditionally inject the option, but this adds complexity.
- Alternatively, make it a configurable flag in the model wrapper (e.g., `include_usage_in_stream: true`), but that still shifts responsibility to the user.

**Additional context**
This issue commonly occurs with Alibaba Cloud’s Bailian API and potentially other OpenAI-compatible services that follow the newer OpenAI API spec where `include_usage` is opt-in for streaming responses. Without this option, the final message with `"usage"` is omitted, causing AgentScope to miss critical token usage metrics. Making this behavior default (or smartly inferred) would improve compatibility and user experience.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.