Azure / Azure/azure-sdk-for-python
Sometimes AI Search Token Count is Missing on Last Query
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
- **Package Name**: azure-ai-projects
- **Package Version**: 1.0.0b12
- **Operating System**: MacOS 15.6
- **Python Version**: 3.11.13
**Describe the bug**
Using the stream API and Azure AI Search tool, sometimes the last query contains "0" for prompt_tokens and completion_tokens (as shown below). This is probably about 15% of the time; most of the time it does show the tokens. The `thread.run.completed` message still totals what is shown in the other messages so we cannot simply subtract to find the missing tokens. I don't think it is possible that this query could be executed with 0 tokens, but if I am wrong, that would be good to know too.
('thread.run.step.completed', {'id': 'step_h5AgeGyvfxpZ77fe611fo5Yx', 'object': 'thread.run.step', 'created_at': 1756251982, 'run_id': 'run_jXFEuKYyr8NVw4SVz5XJqjLy', 'thread_id': 'thread_dHHkFHDIhNZr8LdhvqCyyNYN', 'type': 'tool_calls', 'status': 'completed', 'completed_at': 1756251982, 'step_details': {'type': 'tool_calls', 'tool_calls': [{'id': 'call_o2hi7bRtztTcn4FAOdWqTHn9', 'type': 'azure_ai_search', 'azure_ai_search': {'input': '{"query":"*redacted*"}', 'output': "{'summary': 'Retrieved 5 documents.', 'metadata': {'urls': ['*redacted*', '*redacted*', '*redacted*', '*redacted*', '*redacted*'], 'titles': ['*redacted*', '*redacted*', '*redacted*', '*redacted*', '*redacted*'], 'get_urls': ['*redacted*', '*redacted*', '*redacted*', '*redacted*', '*redacted*'], 'ids': ['*redacted*', '*redacted*', '*redacted*', '*redacted*', '*redacted*'], 'command': 'search', 'query_type': 'simple', 'top_k': 5, 'filter': True}}"}}]}, 'usage': {'prompt_tokens': 0, 'completion_tokens': 0, 'total_tokens': 0, 'prompt_token_details': {'cached_tokens': 0}}, 'expired_at': 1756253778, 'assistant_id': 'asst_HIrSdd5sJCEE08NWPCShmsdl'}, None)
**To Reproduce**
Steps to reproduce the behavior:
1. Create an Agent with an Azure AI Search tool
2. Create a simple prompt to query the tool
3. Run it a bunch of times
4. Observe that on some executions, the last token count is missing
**Expected behavior**
All tool calls to Azure AI Search should include a token count
**Screenshots**
NA
**Additional context**
I am a Microsoft employee and can be reached by `pelasne` on Teams.
Contributor guide
Assessment
This issue has not been assessed yet.