open-telemetry / open-telemetry/opentelemetry-python-genai

[opentelemetry-util-genai] gen_ai.tool.definitions keeps description and parameters when content capture is off

Open
#655 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

Describe your environment
OS: Ubuntu
Python version: Python 3.12.14
Package version: opentelemetry-util-genai 1.2b0.dev (main, 3e6a57d)
GenAI library (e.g. anthropic, openai) and version: n/a, reproduces through the util alone
What happened?

get_content_attributes in util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py
takes an early return when content capture is disabled. Its comment says the intent is to emit the
attribute while leaving the optional properties out:

# Tool definitions are always captured, the sem conv recommends adding params / description only
# when the content capture mode is set..
if mode not in allowed_modes:
    return (
        {GenAI.GEN_AI_TOOL_DEFINITIONS: serialize(tool_definitions)}
        if tool_definitions
        else {}
    )

serialize() is the same full asdict() serialization the capture-enabled path uses, so
description and parameters go out either way. The code does not do what its comment says.

model/gen-ai/gen-ai-tool-definitions.json in semantic-conventions-genai requires only type and
name. Both the schema and the gen_ai.tool.definitions entry in model/gen-ai/registry.yaml say:

Since this attribute could be large, it's NOT RECOMMENDED to populate non-required properties by
default. Instrumentations MAY provide a way to enable populating optional properties.

parameters holds the tool's full JSON Schema, so it is the large part of the payload, and the
registry entry also carries a "may contain sensitive information" warning. Someone who disables
content capture still receives it.

Steps to Reproduce
import os

os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "NO_CONTENT"

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
    InMemorySpanExporter,
)
from opentelemetry.util.genai.handler import TelemetryHandler
from opentelemetry.util.genai.types import FunctionToolDefinition

exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))

handler = TelemetryHandler(tracer_provider=provider)
invocation = handler.inference(provider="anthropic", request_model="a-model")
invocation.tool_definitions = [
    FunctionToolDefinition(
        name="get_weather",
        description="Get weather by city",
        parameters={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    )
]
invocation.stop()

span = exporter.get_finished_spans()[0]
print(span.attributes["gen_ai.tool.definitions"])
Expected Result

Only the properties the schema requires:

[{"name":"get_weather","type":"function"}]
Actual Result
[{"name":"get_weather","description":"Get weather by city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]},"type":"function"}]
Additional context

There is a second question behind this one, and answering it first may change the fix.

model/gen-ai/spans.yaml marks gen_ai.tool.definitions requirement_level: opt_in and puts it in
the attributes.gen_ai.content group next to gen_ai.input.messages, gen_ai.output.messages and
gen_ai.system_instructions. That placement reads as "the attribute is content and follows the
content-capture setting", which is the opposite of "always captured".

The instrumentations in this repo are split on it. genai-openai gates tool_definitions behind
capture_content in both utils.py (Chat Completions) and response_extractors.py (Responses API).
genai-langchain, genai-agno, genai-portkey and genai-smolagents set it regardless. The same
call therefore yields a different attribute depending on which provider you use.

Suggested order:

  1. Decide whether the attribute follows the content-capture setting, given opt_in and its
    membership in the content attribute group.
  2. Either way, make the capture-disabled path serialize only type and name rather than the whole
    object, so the code matches its comment and the semconv guidance.
  3. Align the instrumentations on the outcome.

References:

Would you like to implement a fix?

No

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with get_content_attributes in util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py and read the referenced semantic-convention files to resolve whether tool definitions follow content capture. Then inspect tool_definitions handling in genai-openai/utils.py, response_extractors.py, genai-langchain, genai-agno, genai-portkey, and genai-smolagents. Done means the capture-disabled behavior and instrumentation outcomes are consistently aligned with the chosen convention and the supplied reproduction no longer emits optional fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability-sre
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.