open-telemetry / open-telemetry/opentelemetry-python-genai

instrumentation-genai-langchain: ChatOllama produces no inference span (model key is ls_model_name) and tool-call output parts are dropped when the provider reports done_reason

Open
#679 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

Version: opentelemetry-instrumentation-genai-langchain 1.1b1
(callback_handler.py:214-227 and :322-388), with langchain-ollama 1.1.0.

What happens.

  1. on_chat_model_start resolves the request model from invocation_params and metadata under
    the keys model_name, model_id, model, and returns without a span when none is present.
    ChatOllama publishes the model only as the LangSmith key metadata["ls_model_name"], so no
    inference span is ever produced for it (tool and chain spans are unaffected).
  2. on_llm_end builds ToolCallRequest output parts only when the resolved finish_reason is
    tool_calls or tool_use. ChatOllama reports done_reason in generation_info (no
    finish_reason), so the resolved value is "unknown" and the message's tool_calls are
    dropped from gen_ai.output.messages; downstream, tool spans cannot be linked to the
    inference that requested them.

Reproduce. ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation.

Suggested fix. Fall back to metadata["ls_model_name"] (and ls_provider, which
normalize_provider already reads) when no model key is present; build tool-call parts whenever
message.tool_calls is non-empty, independent of the provider's finish-reason spelling.

Downstream workaround (forgehawk SDK). ForgehawkLangChainCallbackHandler copies ls_model_name into
metadata["model"] and passes the official on_llm_end a view with finish_reason="tool_calls"
when the message carries tool calls and the official resolution found nothing
(sdk/python/src/forgehawk/instrumentors/langchain.py).


Found while building the forgehawk Python SDK (ZioSec) on top of opentelemetry-util-genai; happy to open a PR for the suggested fix if maintainers agree with the direction.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read callback_handler.py:214-227 and 322-388, then reproduce with ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation. Done means the invocation produces an inference span using the available metadata and preserves non-empty message.tool_calls in gen_ai.output.messages even when generation_info reports done_reason; no test file is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.