deepset-ai / deepset-ai/haystack-core-integrations
bug(langfuse): content tracing fails Agent runs when a tool returns None, a string, or an image
@sjrl is already working on this.
Since Sep 15, 2026.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
Describe the bug
With HAYSTACK_CONTENT_TRACING_ENABLED=true and LangfuseConnector, LangfuseSpan.set_content_tag can raise inside the traced run, so the whole Agent or Pipeline run fails. With tracing off, the same run succeeds.
Since Haystack 3.0 (deepset-ai/haystack#11892), the Agent creates one span per tool call and passes the raw tool arguments and result to set_content_tag. The Langfuse tracer assumes that every *.input value containing "messages" is a dict of ChatMessages, and that every *.output value containing "replies" is a dict. Values that don't fit this break it:
| Value traced | Error |
|---|---|
Tool returns None, an int, a float or a bool |
TypeError: argument of type 'NoneType' is not iterable (or 'int' etc.) |
Tool returns a string containing "replies", e.g. "No replies found" |
AttributeError: 'str' object has no attribute 'get' |
Tool returns an image (outputs_to_string={"raw_result": True}) |
ValueError: Unsupported tool result from ChatMessage.to_openai_dict_format, raised on the next LLM step's input |
A component input named messages that isn't list[ChatMessage], e.g. list[str] |
AttributeError: 'str' object has no attribute 'to_openai_dict_format' |
To Reproduce
This needs only haystack-ai and langfuse-haystack. Any Langfuse keys work: the error doesn't depend on spans actually being exported. A scripted chat generator stands in for a real one so that no LLM API key is needed. OpenAIChatGenerator behaves the same way.
import os
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
os.environ.setdefault("LANGFUSE_PUBLIC_KEY", "pk-lf-...")
os.environ.setdefault("LANGFUSE_SECRET_KEY", "sk-lf-...")
from haystack import component
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.tools import tool
from haystack_integrations.components.connectors.langfuse import LangfuseConnector
@component
class ScriptedChatGenerator:
"""Calls the tool once, then answers."""
def __init__(self):
self.calls = 0
@component.output_types(replies=list[ChatMessage])
def run(self, messages: list[ChatMessage], tools=None, **kwargs):
self.calls += 1
if self.calls == 1:
return {"replies": [ChatMessage.from_assistant(tool_calls=[ToolCall(tool_name="notify", arguments={"to": "bob"}, id="call_1")])]}
return {"replies": [ChatMessage.from_assistant("Done")]}
@tool
def notify(to: str) -> None:
"""Send a notification to a user."""
LangfuseConnector("repro").warm_up()
agent = Agent(chat_generator=ScriptedChatGenerator(), tools=[notify])
print(agent.run(messages=[ChatMessage.from_user("Tell Bob")])["last_message"].text)
# -> TypeError: argument of type 'NoneType' is not iterable
Expected behavior
Tracing never makes a run fail. Values that aren't chat messages are recorded with tracing_utils.coerce_tag_value, like other tags.
Describe your environment (please complete the following information):
- OS: macOS
- Haystack version: haystack-ai 3.1.1
- Integration version: langfuse-haystack 6.0.0 (also
main); langfuse 4.15.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.