open-telemetry / open-telemetry/opentelemetry-python-genai
google-genai: instrument_generate_content() unconditionally sets OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT=true, overriding config and user setting
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
Package
opentelemetry-instrumentation-google-genai 1.0b0 (with opentelemetry-util-genai 1.0b0)
What happens
instrument_generate_content() unconditionally mutates a global process environment variable when instrumenting:
def instrument_generate_content(
telemetry_handler: TelemetryHandler,
generate_content_config_key_allowlist: AllowList,
) -> object:
os.environ["OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT"] = "true"
...
This has a few problems:
-
It overrides the documented default behavior.
opentelemetry.util.genai.utils.should_emit_event()is documented to default based onOTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT:NO_CONTENTorSPAN_ONLY→ events default offEVENT_ONLYorSPAN_AND_EVENT→ events default on
Because instrumenting force-sets
OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT=true(whichshould_emit_event()treats as highest priority), thegen_ai.client.inference.operation.detailsevent is emitted for every request even inNO_CONTENT/SPAN_ONLYmode. In those modes the event carries no message content, so it just duplicates the span's attributes — redundant telemetry the user did not ask for. -
It overrides an explicit user setting. A user who sets
OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT=falseto disable the event has their choice silently overwritten byinstrument(). -
It mutates global process state as a side effect of instrumenting. Writing to
os.environaffects the whole process (and any other GenAI instrumentation reading the same variable), is order-dependent, and is not thread-safe.
Minimal reproduction
import os
os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "SPAN_ONLY"
os.environ["OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT"] = "false" # explicitly disable the event
from opentelemetry.instrumentation.google_genai import GoogleGenAiSdkInstrumentor
GoogleGenAiSdkInstrumentor().instrument()
print(os.environ["OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT"])
# -> "true" (the user's "false" was overwritten)
After this, a generate_content call emits a gen_ai.client.inference.operation.details event even though the user asked for SPAN_ONLY capture and explicitly set EMIT_EVENT=false.
Expected
instrument() should not write to os.environ. Whether the event is emitted should be decided by should_emit_event() from the (unmodified) configuration:
- default off for
NO_CONTENT/SPAN_ONLY, - default on for
EVENT_ONLY/SPAN_AND_EVENT, - and an explicit
OTEL_INSTRUMENTATION_GENAI_EMIT_EVENTfrom the user always respected.
If there is a reason the event must be emitted regardless of capture mode, it would be clearer to pass that intent into the TelemetryHandler / invocation directly rather than mutating a global environment variable.
Context
Found while integrating this instrumentation into Pydantic Logfire. As a downstream workaround we snapshot OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT before calling instrument() and restore it afterwards, but that only works because should_emit_event() reads the variable per request rather than at instrument time.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in instrumentation/opentelemetry-instrumentation-google-genai/src/opentelemetry/instrumentation/google_genai/generate_content.py at instrument_generate_content(), then trace how should_emit_event() reads configuration. Run the existing Google GenAI instrumentation tests and add coverage for explicit false and SPAN_ONLY or NO_CONTENT settings. Done means instrument() leaves os.environ unchanged and event emission follows the documented configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100