Azure / Azure/azure-sdk-for-python
No usage tokens in Deep Research tool output
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
The problem is the following: **The run's steps returned once the Deep Research job is done contain (almost) NO useful information regarding token consumption of o3-deep-research model nor BING calls.**
When one implements the deep research agent using the SDK as it is explained in the tutorial:
https://learn.microsoft.com/en-gb/azure/ai-foundry/agents/how-to/tools/deep-research-samples?pivots=python
and ones wants to get details regarding the tools usage (like BING calls or o3-deep-research token usage, not just the gpt-4o ones) one can simple do:
`run_steps = ai_project_client.agents.run_steps.list(thread_id=thread_id, run_id=run_id)`
and inspect each step in run_steps, but **NONE of those steps contain BING calls neither token usage for the o3-deep_research model**
The response instead contains a bunch of steps with ZERO token usage for message_creation type (gpt-4o), like this
`"usage": {
"prompt_tokens": 0,
"completion_tokens": 0,
"total_tokens": 0,
"prompt_token_details": {
"cached_tokens": 0
}
}`
and one last step which contains 1 call tool to deep_research and some token consumption, something that looks like this:
`"step_details": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_SOME_ID",
"type": "deep_research",
"deep_research": {
"input": "{\"prompt\": \"SOME PROMPT\"}",
"output": null
}
}
]
},
"usage": {
"prompt_tokens": 4055,
"completion_tokens": 268,
"total_tokens": 4323,
"prompt_token_details": {
"cached_tokens": 0
}
}`
and after checking the Azure portal, I could confirmed that those tokens **ARE NOT** for the o3-deep-research model, but exactly the ones for gpt-4o.
**Expected Behaviour**: I would expect to get in those steps some with Non zero usage for gpt-4o calls and also some having BING calls (and of course the real token values usage for o3-deep-research model, which I can see in the UI and are definitely not the ones appearing in the corresponding step)
PD: Just for the record, inspecting those steps **does work**** in other examples, like for instance this one
https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-agents/samples/agents_tools/sample_agents_bing_grounding.py
which makes me believe there is a bug for the Deep Research module
Contributor guide
Assessment
This issue has not been assessed yet.