open-telemetry / open-telemetry/opentelemetry-python-genai
The hook functions in LangChain middleware, such as before_agent, are not wrapped
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
Describe your environment
OS: (e.g, Ubuntu)
Python version: (e.g., Python 3.12.0)
Package version: (e.g., 0.5b0)
GenAI library (e.g. anthropic, openai) and version:
What happened?
When opentelemetry-instrumentation-langchain is enabled, LangChain middleware hooks (before_agent, before_model, after_model, after_agent and their async variants) that are overridden by subclasses never get registered into the LangGraph state graph, causing them to never execute.
Steps to Reproduce
opentelemetry-instrument python test.py
import asyncio
from dotenv import load_dotenv
load_dotenv()
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware
from langchain.agents.middleware.types import AgentState, ModelRequest, ModelResponse
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os
openai_model: str = os.environ.get("OPENAI_MODEL", "")
openai_base_url: str = os.environ.get("OPENAI_BASE_URL", "")
openai_api_key: str = os.environ.get("OPENAI_API_KEY", "")
CALL_LOG = []
class TestMiddleware(AgentMiddleware):
def before_agent(self, state, runtime):
CALL_LOG.append("before_agent called")
print(">>> before_agent called")
return None
async def awrap_model_call(self, request, handler):
CALL_LOG.append("awrap_model_call called")
print(">>> awrap_model_call called")
return await handler(request)
@tool
def dummy_tool(query: str) -> str:
"""A dummy tool that returns the query."""
return f"Result: {query}"
async def main():
llm = ChatOpenAI(
model=openai_model,
base_url=openai_base_url,
api_key=openai_api_key, # type: ignore
temperature=0
)
agent = create_agent(
model=llm,
tools=[dummy_tool],
system_prompt="You are a helpful assistant.",
middleware=[TestMiddleware()],
name="test_agent",
)
result = await agent.ainvoke(
{"messages": [{"role": "user", "content": "Hello"}]},
config={"configurable": {"thread_id": "test-1"}},
)
print(f"\nCALL_LOG: {CALL_LOG}")
print(f"before_agent was called: {'before_agent called' in CALL_LOG}")
print(f"awrap_model_call was called: {'awrap_model_call called' in CALL_LOG}")
asyncio.run(main())
Expected Result
awrap_model_call called
CALL_LOG: ['awrap_model_call called']
before_agent was called: True
awrap_model_call was called: True
Actual Result
awrap_model_call called
CALL_LOG: ['awrap_model_call called']
before_agent was called: False
awrap_model_call was called: True
Additional context
No response
Would you like to implement a fix?
None
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided test.py with opentelemetry-instrument and compare the before_agent and awrap_model_call results. Trace the LangChain middleware instrumentation entry point that registers hooks with create_agent, then verify that overridden synchronous and asynchronous hooks execute for the reproduction case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100