open-telemetry / open-telemetry/opentelemetry-python-genai

langchain: dict-form messages raise in on_chain_start and silently drop the span

Open
#388 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

Passing chat messages as plain dicts, which LangChain accepts everywhere and which its own docs use, raises inside on_chain_start. LangChain catches callback exceptions and logs them, so the failure is silent: the workflow or agent span is simply never created, and only the child chat spans survive.

Repro

from langchain.agents import create_agent
from langchain_openai import ChatOpenAI

agent = create_agent(
    model=ChatOpenAI(model="gpt-4o-mini", max_tokens=100),
    tools=[],
    system_prompt="You are a helpful assistant.",
    name="weather_assistant",
)
agent.invoke({"messages": [{"role": "user", "content": "hi"}]})

Run under opentelemetry-instrument.

Actual:

WARNING langchain_core.callbacks.manager: Error in
OpenTelemetryLangChainCallbackHandler.on_chain_start callback:
AttributeError("'dict' object has no attribute 'content'")

and no chain-level span at all. Replacing the dict with
HumanMessage(content="hi") produces the span, so the message form is the only
difference.

Expected: the same telemetry either way. A supported input form should not
silently remove a span.

Why it happens

on_chain_start calls make_input_message(inputs) (utils.py). When
inputs["messages"] exists it is passed straight to to_input_messages, which
is annotated Iterable[BaseMessage] and reads .content off each entry. The
cast is unchecked, so a dict entry raises.

make_input_message already has a fallback for state without a messages key.
The same tolerance is needed per entry: convert dicts with
langchain_core.messages.convert_to_messages, or skip entries that are not
BaseMessage rather than raising.

make_output_message casts identically, so on_chain_end is exposed to the
same failure for outputs.

Context

Found while adding LangChain conformance scenarios in
https://github.com/open-telemetry/semantic-conventions-conformance/pull/33.
Related to #387, which is about the operation the span is classified as; this
one is about the span not existing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in utils.py with make_input_message and to_input_messages, then inspect make_output_message and the on_chain_start/on_chain_end callback paths. Reproduce the issue with the provided agent invocation under opentelemetry-instrument and compare it with HumanMessage input. Done means dict-form messages produce the same chain-level telemetry without callback exceptions or dropped spans.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.