The new opt-out style mechanism of OTEL_PYTHON_EXCLUDED_URLS etc. is INHERENTLY INSECURE

Describe your environment

OS: (e.g, Ubuntu) - MacOS and Linux
Python version: 3.11.13
Package version: 0.59b0

What happened?

The new opt-out style mechanism involving environment variables OTEL_PYTHON_REQUESTS_EXCLUDED_URLS, OTEL_PYTHON_EXCLUDED_URLS, and OTEL_PYTHON_AIOHTTP_CLIENT_EXCLUDED_URLS - is inherently insecure.

One of the core purposes of these mechanisms is to prevent unwanted leakage of sensitive data to 3rd party targets. However, an opt-out mechanism such as the above is error-prone and insecure.

Due to the cross-cutting nature of opentelemetry instrumentation, it's very easy to miss some new functionality that makes REST API calls to a 3rd party being added in a larger application, where a developer is unaware of the need to exclude certain URLs from propagation. Or there might even be some minor misspelling in the opt-out, etc. As a result, potentially sensitive trace and baggage data may be exposed to 3rd party targets for a long time before being discovered.

Steps to Reproduce

Developer misses or misspells a URL that should go into OTEL_PYTHON_EXCLUDED_URLS or one of the related environment variables. This results in potentially sensitive trace and/or baggage data being exposed to 3rd party targets for a long time before being discovered, if ever!

Expected Result

The mechanism needs to provide applications a robust way to explicitly control where their potentially sensitive data goes.

1. One possibility is to make it an opt-in (i.e. whitelisting) mechanism. Or a combination of otp-in/opt-out where a URL MUST pass opt-in first before being subject to opt-out. If opt-in configuration is missing, then nothing should be propagated to any targets.
2. Furthermore, while preventing trace and/or garbage propagation to unwanted targets, the mechanism still needs to allow a trace span to be created so that we can observe the span in our graphs. Judging by current implementation, if I am not mistaken, it looks like if a URL is excluded, then a span isn't created either.
3. There should also be a programmatic way to configure opt-in/opt-out.
4. Finally, consider a use case where only baggage contains sensitive information, so we may need to block baggage propagation but still allow trace propagation to certain targets of end-to-end cross-system debugging.

Actual Result

The currently-implemented mechanism is easily error-prone, allowing potentially sensitive information to be exposed to unwanted targets.

Additional context

No response

Would you like to implement a fix?

None

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

Start by reviewing instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/init.py and the handling of OTEL_PYTHON_EXCLUDED_URLS and related variables. Define how explicit opt-in, programmatic configuration, span creation, and separate trace and baggage propagation should work before locating affected instrumentation and tests. Done means sensitive data is not propagated to unapproved targets while permitted spans remain observable.