open-telemetry / open-telemetry/opentelemetry-python-genai
langchain: dict-form messages raise in on_chain_start and silently drop the span
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
Passing chat messages as plain dicts, which LangChain accepts everywhere and which its own docs use, raises inside on_chain_start. LangChain catches callback exceptions and logs them, so the failure is silent: the workflow or agent span is simply never created, and only the child chat spans survive.
Repro
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
agent = create_agent(
model=ChatOpenAI(model="gpt-4o-mini", max_tokens=100),
tools=[],
system_prompt="You are a helpful assistant.",
name="weather_assistant",
)
agent.invoke({"messages": [{"role": "user", "content": "hi"}]})
Run under opentelemetry-instrument.
Actual:
WARNING langchain_core.callbacks.manager: Error in
OpenTelemetryLangChainCallbackHandler.on_chain_start callback:
AttributeError("'dict' object has no attribute 'content'")
and no chain-level span at all. Replacing the dict with
HumanMessage(content="hi") produces the span, so the message form is the only
difference.
Expected: the same telemetry either way. A supported input form should not
silently remove a span.
Why it happens
on_chain_start calls make_input_message(inputs) (utils.py). When
inputs["messages"] exists it is passed straight to to_input_messages, which
is annotated Iterable[BaseMessage] and reads .content off each entry. The
cast is unchecked, so a dict entry raises.
make_input_message already has a fallback for state without a messages key.
The same tolerance is needed per entry: convert dicts with
langchain_core.messages.convert_to_messages, or skip entries that are not
BaseMessage rather than raising.
make_output_message casts identically, so on_chain_end is exposed to the
same failure for outputs.
Context
Found while adding LangChain conformance scenarios in
https://github.com/open-telemetry/semantic-conventions-conformance/pull/33.
Related to #387, which is about the operation the span is classified as; this
one is about the span not existing.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in utils.py with make_input_message and to_input_messages, then inspect make_output_message and the on_chain_start/on_chain_end callback paths. Reproduce the issue with the provided agent invocation under opentelemetry-instrument and compare it with HumanMessage input. Done means dict-form messages produce the same chain-level telemetry without callback exceptions or dropped spans.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100