open-telemetry / open-telemetry/opentelemetry-java-instrumentation
Add include/exclude selectors for captured collections
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1.2k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 228
Description
Is your feature request related to a problem? Please describe.
Capture collection settings are allowlists only, and action-oriented names do not extend naturally to exclusion: capture_mdc_attributes.excluded is contradictory. Java APIs need the same selector model without separate, verbose include/exclude methods. The selector implementation should also be shared with the Java SDK and contrib where practical instead of creating parallel public APIs with the same semantics.
Describe the solution you'd like
Adopt an immutable public IncludeExclude value with declarative configuration's exact and wildcard matching semantics. Exclusions apply after inclusions. The value exposes its configured patterns, reports whether it is empty, and can test whether a string matches.
Phase 0: Establish the shared selector API
Status: resolved as the fallback. opentelemetry-java#7599 is still open and core maintainers leaned against publishing the type from the SDK, so the instrumentation-local implementation shipped in #19451 as io.opentelemetry.instrumentation.api.config.IncludeExclude in instrumentation-api. Moving it to core remains possible if the upstream discussion concludes differently.
Resolve ownership of the shared Java type before migrating instrumentation settings. Prefer defining the public type in the OpenTelemetry Java core repository if its maintainers agree, so the SDK, instrumentation, and contrib repositories can reuse one API without reversing repository dependencies. Continue the existing discussion in opentelemetry-java#7599.
The existing SDK IncludeExcludePredicate has the same core include/exclude semantics but is an internal predicate factory rather than an inspectable configuration value. A shared public API should retain the value-oriented capabilities developed in #19451: immutable included and excluded lists, isEmpty(), value equality, and glob matching. The SDK predicate can delegate its pattern-matching mode to the shared implementation while retaining its separate exact-matching mode.
#19451 currently prototypes the type as io.opentelemetry.instrumentation.api.config.IncludeExclude. If the core maintainers accept the shared API, move the implementation to core and make this repository consume it. If core declines to own it, retain the instrumentation-local implementation as the fallback.
Each setting defines its default value
IncludeExclude has one matching rule independent of the setting that uses it: when there are no included patterns, all values that are not excluded match. The selector does not encode what an unconfigured setting should do. Instead, each setting defines its own default value, which applies when the selector is absent or empty.
An empty selector, one with no patterns in either list, carries no configuration and therefore falls back to the setting's default just like an absent selector. This keeps an empty selector a no-op and matches flat configuration, where empty property values cannot be distinguished from unset ones. Callers use isEmpty() to detect this case.
Default: select no values. This is the default for every capture setting in phase 1. An absent or empty selector selects nothing. Any non-empty selector replaces that default and uses the normal matching rule, so an exclude-only selector selects everything except the excluded values. Select everything explicitly with included: ["*"], or included=* in flat configuration.
Default: select all values. This applies when filtering telemetry that is already emitted by default, such as choosing which default metrics to keep. An absent or empty selector keeps everything. Any non-empty selector uses the same normal matching rule, with an exclude-only selector selecting everything except the excluded values.
The matching meaning of a non-empty selector is therefore identical for every setting; only the value used when the selector is absent or empty varies.
Phase 1: Unify existing non-HTTP collection selectors
Status: complete. All seven non-HTTP collection properties now use noun-based selectors:
-
java.grpc.client.request_metadata.{included,excluded}/setClientRequestMetadata(IncludeExclude)— #19494 -
java.grpc.server.request_metadata.{included,excluded}/setServerRequestMetadata(IncludeExclude)— #19494 -
java.common.messaging.headers/development.{included,excluded}/setHeaders(IncludeExclude)— #19523 -
java.servlet.request_parameters/development.{included,excluded}/setRequestParameters(IncludeExclude)— #19522 -
java.logback_appender.mdc_attributes/development.{included,excluded}/setMdcAttributes(IncludeExclude)— #19520 -
java.log4j_appender.mdc_attributes/development.{included,excluded}/setContextDataAttributes(IncludeExclude)— #19521 -
java.jboss_logmanager.mdc_attributes/development.{included,excluded}/ no public Java API — #19519
JFR runtime metrics also adopted the selector in #19495, which was not part of the original list.
Deprecate old properties and Java setters, treating old collections as include-only aliases through 2.x, then remove them in 3.0. Update schemas, metadata, documentation, examples, and migration guidance.
Optional phase 2: Extend selectors to keyed logging maps
Status: not started. All five settings are still booleans.
Optionally convert the five boolean-gated Log4j and Logback settings for dynamic keyed maps. Legacy true maps to an explicit select-all selector, while false maps to an absent selector whose setting default selects no values.
-
java.log4j_appender.map_message_attributes/development.{included,excluded}/setMapMessageAttributes(IncludeExclude) -
java.logback_appender.key_value_pair_attributes/development.{included,excluded}/setKeyValuePairAttributes(IncludeExclude) -
java.logback_appender.logger_context_attributes/development.{included,excluded}/setLoggerContextAttributes(IncludeExclude) -
java.logback_appender.logstash_marker_attributes/development.{included,excluded}/setLogstashMarkerAttributes(IncludeExclude) -
java.logback_appender.logstash_structured_argument_attributes/development.{included,excluded}/setLogstashStructuredArgumentAttributes(IncludeExclude)
Other capture booleans remain unchanged. JDBC query parameters stay boolean because their positional keys make filtering brittle.
Phase 3: Coordinate HTTP header selectors upstream
Status: not started. The four HTTP header properties are unchanged.
Defer the four HTTP properties until coordinating the declarative shape upstream. HTTP semantic conventions SHOULD require explicit configuration because capture-all can leak sensitive data, but MAY allow users to configure all headers. They do not prohibit wildcards.
The current declarative schema accepts arbitrary strings but defines no wildcard behavior. Phase 3 should standardize IncludeExclude, define the setting default as selecting no headers, and evaluate any non-empty selector using the shared matching rule. Apply the same 2.x compatibility and 3.0 removal policy as phase 1.
-
general.http.client.request_headers.{included,excluded}/setRequestHeaders(IncludeExclude) -
general.http.client.response_headers.{included,excluded}/setResponseHeaders(IncludeExclude) -
general.http.server.request_headers.{included,excluded}/setRequestHeaders(IncludeExclude) -
general.http.server.response_headers.{included,excluded}/setResponseHeaders(IncludeExclude)
Describe alternatives you've considered
Decision rationale
Use capture_* or captured_* as the selector parent
capture_mdc_attributes.excluded is contradictory, while captured_mdc_attributes.included reads like a filtered result rather than configuration. Naming the resource (mdc_attributes) keeps the parent neutral and lets included and excluded describe selection.
Add paired Java setters
setIncludedRequestHeaders and setExcludedRequestHeaders avoid a new type but split one selector across two mutable calls. A single IncludeExclude value is atomic, avoids call-order and replacement questions, and keeps Java aligned with declarative configuration.
Accept Predicate<String> or expose IncludeExcludePredicate unchanged
The SDK has a stable ViewBuilder.setAttributeFilter(Predicate<String>), backed internally by IncludeExcludePredicate. That filter uses a select-all default, while capture settings use a select-none default. Predicates hide configured patterns, cannot report the empty-selector state, and do not mirror the declarative configuration value. Prefer a public immutable value type that the internal predicate can reuse rather than publishing the current internal class unchanged.
Incubate IncludeExclude in instrumentation-api-incubator
Incubation reduces the initial stability commitment, but stable builders in instrumentation-api cannot expose a type from the incubator module without reversing the dependency direction. Promotion would also require another API transition. If core owns the shared type, it should choose the appropriate core artifact and stability level with the requirement that stable instrumentation APIs can eventually expose it.
Keep an independent implementation in instrumentation-api
This avoids coordination with core and keeps instrumentation independent from SDK artifacts, which is why #19451 started there. It also leaves core, instrumentation, and contrib with parallel implementations of the same configuration concept. Keep this as the fallback if core does not accept a shared public API.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the remaining unchecked Phase 2 and Phase 3 settings, the instrumentation-api IncludeExclude type, and the declarative instrumentation.yaml schema. Review the linked core issue and existing selector work before choosing a scope. Done means completing a clearly scoped selector migration with compatibility, schema, metadata, documentation, and tests updated as applicable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100