traceloop / traceloop/openllmetry

Bug: HTTP CLIENT spans not nested under gen_ai spans (start_span vs start_as_current_span)

Open
#4,365 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.4k
Forks
1.1k
Avg merge
8d 14h
Merged PRs (30d)
2

Description

Description

HTTP CLIENT spans created by opentelemetry-instrumentation-httpx (and other transport instrumentors like opentelemetry-instrumentation-urllib) appear as siblings of anthropic.chat instead of children, because _wrap and _instrumented_converse_stream use tracer.start_span() rather than tracer.start_as_current_span().

When start_span() is used, the span is created but not attached to the OTel context. Any child spans created during the underlying HTTP call (by the httpx instrumentor using start_as_current_span) inherit the ambient context — which is the parent of anthropic.chat, not anthropic.chat itself.

Expected trace tree

invoke_agent
└── anthropic.chat          ← gen_ai span
    └── POST /v1/messages   ← HTTP CLIENT span (currently a sibling, not a child)

Actual trace tree

invoke_agent
├── anthropic.chat          ← gen_ai span
└── POST /v1/messages       ← HTTP CLIENT span (sibling instead of child)

Root cause

In opentelemetry-instrumentation-anthropic, _wrap (line ~555):

span = tracer.start_span(name, kind=SpanKind.CLIENT, attributes={...})
# span is NOT the current context from here
...
response = wrapped(*args, **kwargs)  # HTTP call happens here — wrong parent

In opentelemetry-instrumentation-bedrock, _instrumented_converse_stream (line ~334):

span = tracer.start_span(...)
# span is NOT the current context
try:
    response = fn(*args, **kwargs)  # boto3/httpx call — wrong parent

Note: _instrumented_converse (non-streaming) correctly uses start_as_current_span — so the nesting bug only affects the streaming path in the bedrock instrumentor and all paths in the anthropic instrumentor.

Minimal fix

Attach the span to context for the duration of the underlying call, then detach (keeping the span open for streaming):

# anthropic _wrap — around response = wrapped(*args, **kwargs)
ctx_token = context_api.attach(context_api.set_value(SUPPRESS_LANGUAGE_MODEL_INSTRUMENTATION_KEY, True))
# ACTUALLY: attach the span itself
from opentelemetry.trace import set_span_in_context
ctx_token = context_api.attach(set_span_in_context(span))
try:
    response = wrapped(*args, **kwargs)
except Exception as e:
    ...
    raise
finally:
    context_api.detach(ctx_token)

Same pattern in _instrumented_converse_stream.

context.attach + detach is safe for streaming: it makes the span current during the HTTP handshake (so child spans nest correctly) but detaches immediately after wrapped() returns, while the span itself remains open for the streaming body to be consumed.

Environment

  • opentelemetry-instrumentation-anthropic 0.62.1
  • opentelemetry-instrumentation-bedrock 0.62.1
  • Python 3.12

Impact

With opentelemetry-instrumentation-httpx installed alongside these instrumentors, all HTTP CLIENT spans emitted during LLM calls appear at the wrong level in the trace tree, making it impossible to correlate the HTTP request latency and status code with the specific LLM call that triggered it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting _wrap in opentelemetry-instrumentation-anthropic and _instrumented_converse_stream in opentelemetry-instrumentation-bedrock; compare the latter with _instrumented_converse, which already uses the current-span pattern. Verify that HTTP CLIENT spans nest under the gen_ai span during the underlying call, while streaming spans remain open after the context is detached.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.