open-telemetry / open-telemetry/opentelemetry-python-genai

openai: embeddings path never honors OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT

Open
#379 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

What I observed

The OpenAI embeddings example documents message-content capture, but the embeddings code path never consults the content-capture setting.

In instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch.py, the chat paths read the setting and thread it through:

  • chat_completions_createcapture_content = handler.should_capture_content() (line 65), passed to _create_chat_invocation(...) and _set_response_properties(...)
  • the async variant does the same at line 96

The embeddings path does not:

  • embeddings_create (line 124) calls _create_embedding_invocation(handler, kwargs, instance) with no capture_content argument
  • _create_embedding_invocation (lines 34-58) records only request_model, server_address, server_port, dimension_count, and encoding_formats. The embedding input is never recorded on the span or emitted as an event.

Meanwhile instrumentation/opentelemetry-instrumentation-genai-openai/examples/embeddings/README.rst tells the user that setting OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT "configures OpenAI instrumentation to capture content on events", and the accompanying .env sets it. Following that example produces no content for embeddings regardless of the value.

Observed on main at 15d92a7.

Question

Is this intentional — embedding inputs are deliberately never captured, in which case the embeddings example's documentation is the thing that is wrong — or is it a gap in the embeddings path?

I am happy to send a PR either way once you tell me which direction you prefer:

  1. Adjust the embeddings example documentation so it does not advertise content capture, or
  2. Thread capture_content through the embeddings path so the setting is honored.

I did not want to guess at the intended semantics for embedding inputs, since capturing them has different privacy characteristics from chat prompts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch.py, comparing embeddings_create and _create_embedding_invocation with the chat paths' content-capture handling. Review instrumentation/opentelemetry-instrumentation-genai-openai/examples/embeddings/README.rst and its .env to determine which behavior is intended. Done means the implementation and example documentation consistently describe and honor the chosen content-capture behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.