microsoft / microsoft/agent-framework
Python: [Bug]: Token Count Discrepancy between Microsoft Foundry and Agent Framework - Potential Billing Issue
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
### Description
Billing and Token Usage Discrepancy in Microsoft Foundry Agent vs. Agent Framework
We have a Microsoft Foundry Agent with tool calls.
We've rebuilt this Agent inside the Agent Framework using the Harness and ran it locally.
Both Agents have the exact same behavior. Same traces, same tool calls, same results, same system prompt.
### Microsoft Foundry Agent Call:
- Microsoft Foundry reports an input token count of 35k
### Agent Framwork Local Agent Call:
- UsageDetails reports `INFO __main__: Token usage (turn): in=144378 out=831 total=145209 reasoning=269 cached_in=105887`
### In the Corresponding `Model Monitor` in Foundry:
**Before the Calls:**
- Total requests: 0
- Total token count: 0
- Avg per request: 0
**After the Foundry Agent Call**
- Total requests: 1
- Total token count: 35k
- Avg per request: 35
**After the Agent Framework Agent Call (after the Foundry Agent Call)**
- Total requests: 1 (did not get logged as request should be 2)
- total token count: 180k (145k+35k)
- Avg per request: 180k
Our system prompt has 14k tokens.
It seems that the Agent Framework forwards the entire history to the agent after each tool call.
This seems not to be the case for the Foundry Agent.
Or the tokens are counted differently.
If so, then the Foundry Agent probably only counts the non cached input tokens and reports them to the `Model Monitor`.
While the Agent Framework reports the entire token count including the cached input tokens to the `Model Monitor`.
Unsure how this discrepancy affects billing in the agent framework compared to foundry.
If this is intended, a short explanation about how this affects billing would be appreciated.
Thanks in advance.
### Code Sample
```markdown
session_usage: UsageDetails | None = None
agent = create_harness_agent(
client=client,
name="productsearch-agent",
agent_instructions=_load_system_prompt(),
tools=[get_facets, search_field],
default_options={"store": True, "allow_multiple_tool_calls": False, "response_format": ProductRecommendations},
disable_file_memory=True,
middleware=[UsageLoggingMiddleware()],
auto_approval_rules=[SkillsProvider.read_only_tools_auto_approval_rule],
)
session = agent.create_session()
result = await agent.run(user_input, session=session)
session_usage = add_usage_details(session_usage, result.usage_details)
```
### Error Messages / Stack Traces
```markdown
```
### Package Versions
agent-framework: 1.16.0, agent-framework-core: 1.16.0,
### Python Version
Python 3.13
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.