open-telemetry / open-telemetry/opentelemetry-python-genai
[langchain] Capture token usage from llm_output and generation_info fallbacks on chat
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
[!NOTE]
This issue was generated with AI assistance and requires investigation and confirmation prior to being worked on.
Part of #543.
In on_llm_end, token usage is only read from chat_generation.message.usage_metadata.
Older model wrappers, non-streaming outputs, and certain providers (such as Google VertexAI / Gemini) place token usage in response.llm_output["token_usage"]\ (or "usage"for Anthropic) orchat_generation.generation_info["usage_metadata"]. When message.usage_metadata` is absent, token counts are currently recorded as 0 or skipped.
- Support fallback to
response.llm_output.get("token_usage")/response.llm_output.get("usage") - Support fallback to
chat_generation.generation_info.get("usage_metadata") - Extract
gen_ai.usage.input_tokensandgen_ai.usage.output_tokens
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the on_llm_end implementation and trace how chat_generation.message.usage_metadata is currently read. Check the response llm_output and generation generation_info shapes, then verify that the fallback paths extract gen_ai.usage.input_tokens and gen_ai.usage.output_tokens when message metadata is absent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100