open-telemetry / open-telemetry/opentelemetry-python-genai
instrumentation-genai-langchain: ChatOllama produces no inference span (model key is ls_model_name) and tool-call output parts are dropped when the provider reports done_reason
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
Version: opentelemetry-instrumentation-genai-langchain 1.1b1
(callback_handler.py:214-227 and :322-388), with langchain-ollama 1.1.0.
What happens.
on_chat_model_startresolves the request model frominvocation_paramsandmetadataunder
the keysmodel_name,model_id,model, and returns without a span when none is present.
ChatOllamapublishes the model only as the LangSmith keymetadata["ls_model_name"], so no
inference span is ever produced for it (tool and chain spans are unaffected).on_llm_endbuildsToolCallRequestoutput parts only when the resolvedfinish_reasonis
tool_callsortool_use.ChatOllamareportsdone_reasoningeneration_info(no
finish_reason), so the resolved value is"unknown"and the message'stool_callsare
dropped fromgen_ai.output.messages; downstream, tool spans cannot be linked to the
inference that requested them.
Reproduce. ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation.
Suggested fix. Fall back to metadata["ls_model_name"] (and ls_provider, which
normalize_provider already reads) when no model key is present; build tool-call parts whenever
message.tool_calls is non-empty, independent of the provider's finish-reason spelling.
Downstream workaround (forgehawk SDK). ForgehawkLangChainCallbackHandler copies ls_model_name into
metadata["model"] and passes the official on_llm_end a view with finish_reason="tool_calls"
when the message carries tool calls and the official resolution found nothing
(sdk/python/src/forgehawk/instrumentors/langchain.py).
Found while building the forgehawk Python SDK (ZioSec) on top of opentelemetry-util-genai; happy to open a PR for the suggested fix if maintainers agree with the direction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read callback_handler.py:214-227 and 322-388, then reproduce with ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation. Done means the invocation produces an inference span using the available metadata and preserves non-empty message.tool_calls in gen_ai.output.messages even when generation_info reports done_reason; no test file is named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100