open-telemetry / open-telemetry/opentelemetry-python-genai

The hook functions in LangChain middleware, such as before_agent, are not wrapped

Open
#44 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

Describe your environment

OS: (e.g, Ubuntu)
Python version: (e.g., Python 3.12.0)
Package version: (e.g., 0.5b0)
GenAI library (e.g. anthropic, openai) and version:

What happened?

When opentelemetry-instrumentation-langchain is enabled, LangChain middleware hooks (before_agent, before_model, after_model, after_agent and their async variants) that are overridden by subclasses never get registered into the LangGraph state graph, causing them to never execute.

Steps to Reproduce

opentelemetry-instrument python test.py

import asyncio
from dotenv import load_dotenv
load_dotenv()
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware
from langchain.agents.middleware.types import AgentState, ModelRequest, ModelResponse
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

openai_model: str = os.environ.get("OPENAI_MODEL", "")
openai_base_url: str = os.environ.get("OPENAI_BASE_URL", "")
openai_api_key: str = os.environ.get("OPENAI_API_KEY", "")

CALL_LOG = []

class TestMiddleware(AgentMiddleware):
def before_agent(self, state, runtime):
CALL_LOG.append("before_agent called")
print(">>> before_agent called")
return None

async def awrap_model_call(self, request, handler):
    CALL_LOG.append("awrap_model_call called")
    print(">>> awrap_model_call called")
    return await handler(request)
@tool
def dummy_tool(query: str) -> str:
"""A dummy tool that returns the query."""
return f"Result: {query}"

async def main():
  llm = ChatOpenAI(
  model=openai_model,
  base_url=openai_base_url,
  api_key=openai_api_key, # type: ignore
  temperature=0
  )
  agent = create_agent(
  model=llm,
  tools=[dummy_tool],
  system_prompt="You are a helpful assistant.",
  middleware=[TestMiddleware()],
  name="test_agent",
  )
  
  result = await agent.ainvoke(
      {"messages": [{"role": "user", "content": "Hello"}]},
      config={"configurable": {"thread_id": "test-1"}},
  )
  
  print(f"\nCALL_LOG: {CALL_LOG}")
  print(f"before_agent was called: {'before_agent called' in CALL_LOG}")
  print(f"awrap_model_call was called: {'awrap_model_call called' in CALL_LOG}")
asyncio.run(main())
Expected Result

awrap_model_call called
CALL_LOG: ['awrap_model_call called']
before_agent was called: True
awrap_model_call was called: True

Actual Result

awrap_model_call called
CALL_LOG: ['awrap_model_call called']
before_agent was called: False
awrap_model_call was called: True

Additional context

No response

Would you like to implement a fix?

None

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided test.py with opentelemetry-instrument and compare the before_agent and awrap_model_call results. Trace the LangChain middleware instrumentation entry point that registers hooks with create_agent, then verify that overridden synchronous and asynchronous hooks execute for the reproduction case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.