agentscope-ai / agentscope-ai/agentscope
[Bug]: context-compression LLM call's token usage is never recorded anywhere
- Dominant language
- Python
- Stars
- 31.5k
- Forks
- 3.5k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 95
Description
### Prerequisites
- [x] I have searched the existing issues and discussions, and this is not a duplicate.
- [x] This is a bug, not a usage question.
### Background / Description
When an agent's context grows past the compression threshold, AgentScope makes an internal LLM call to summarize/compress the conversation before continuing. That call spends real input/output tokens with the provider, same as any other model call.
Every other model call the agent makes (the normal reasoning/reply calls) ends up with its token usage recorded on the agent's state, so anything built on top of usage tracking (cost estimation, budget limits, per-turn `ModelCallEndEvent` listeners, etc.) sees it. The compression call is the one exception: its usage is never recorded anywhere I can find and never shows up in any event, so any application relying on AgentScope's usage numbers will systematically undercount cost/tokens by the (potentially large, since it summarizes the whole reserved-out context) compression call, with no error or warning.
### Error Messages
Not applicable: this is not a crash or exception. It's a silent gap: the operation completes successfully (the compression happens, the summary is produced and applied), but the token cost of doing so is dropped rather than recorded.
### Steps to Reproduce
Minimal repro using a stub model (no live API needed) that returns a `StructuredResponse` carrying non-zero usage, exactly like a real provider adapter does. It forces compression to trigger on the first check, then compares the usage recorded after compression against the usage recorded after an ordinary model reply with the identical payload:
```python
import asyncio
from pydantic import BaseModel
from agentscope.agent._agent import Agent
from agentscope.model._base import ChatModelBase
from agentscope.model._model_response import StructuredResponse
from agentscope.model._model_usage import ChatUsage
from agentscope.message import TextBlock
class DummyCredential:
pass
class Summary(BaseModel):
task_overview: str = "t"
current_state: str = "s"
important_discoveries: str = "d"
next_steps: str = "n"
context_to_preserve: str = "c"
class StubParameters(BaseModel):
pass
class StubModel(ChatModelBase):
def __init__(self):
super().__init__(
credential=DummyCredential(),
model="stub-model",
parameters=StubParameters(),
stream=False,
context_size=1000,
)
self.structured_calls = 0
async def count_tokens(self, *args, **kwargs) -> int:
return 900 # always over trigger_ratio * context_size
async def generate_structured_output(self, messages, structured_model, **kwargs):
self.structured_calls += 1
return StructuredResponse(
content=Summary().model_dump(),
usage=ChatUsage(input_tokens=12345, output_tokens=678, time=1.23),
)
async def __call__(self, *args, **kwargs):
raise NotImplementedError("not exercised in this repro")
async def main():
model = StubModel()
agent = Agent(name="tester", system_prompt="You are a helpful assistant.", model=model)
agent.state.append_context("user", [TextBlock(type="text", text="hello " * 50)])
agent.state.context[-1].usage = None
await agent._compress_context_impl()
print("structured_output calls:", model.structured_calls)
print("summary set:", bool(agent.state.summary))
print("usage after compression:", agent.state.context[-1].usage if agent.state.context else None)
# Control: an ordinary reply's usage IS recorded on the context tail.
agent._save_to_context(
[TextBlock(type="text", text="a normal reply")],
ChatUsage(input_tokens=12345, output_tokens=678, time=1.23),
)
print("usage after an ordinary reply (control):", agent.state.context[-1].usage)
asyncio.run(main())
```
Output on current `main`:
```
structured_output calls: 1
summary set: True
usage after compression: None
usage after an ordinary reply (control): input_tokens=12345 output_tokens=678
```
The compression call fired (`structured_calls == 1`) and completed successfully (`summary set: True`), but its 12345/678 token usage is nowhere afterward. The identical usage payload attached to an ordinary reply (control) is recorded correctly, so this isn't "usage tracking is absent," it's specific to the context-compression call.
### Environment
- AgentScope Version: 2.0.4.post1 (installed from `main` @ `77318e0`)
- Python Version: 3.12.13
- OS: Linux (Debian, `python:3.12-slim` container)
Contributor guide
Assessment
This issue has not been assessed yet.