open-telemetry / open-telemetry/opentelemetry-cpp
[SDK] Support non-exporting pipeline log record processors (needed for EventToSpanEventBridgeProcessor)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.4k
- Forks
- 632
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 75
Description
Summary
EventToSpanEventBridgeProcessor (proposed in #4309) bridges log-record events onto the live Span referenced by the log record's resolved context. It needs a LogRecordProcessor that can:
- read the resolved context (
Context/liveSpan) atOnEmit()time - read the log record's own data (body, attributes) without taking exclusive ownership away from downstream processors/exporters
The current LogRecordProcessor interface doesn't support this shape of "non-exporting", read/mutate pipeline processor, so the bridge processor can't be implemented correctly without new SDK architecture.
Background
From review on #4309:
Processors like the bridge processor will require some new architectural components and changes to existing components to be spec compliant and maintain reasonable performance. This is in addition to the Logger API level
EmitLogRecordWithContextchange, which is needed.The architecture components / changes that seem to be required include:
- Standardize the SDK built-in exporters (OTLP, ostream) on a recordable with a read-write interface. Pipeline processors need to read data from the record and not duplicate it.
- Create a new processor interface for composable pipeline processors that can read and mutate data but do not own the recordable life-cycle.
- Create a new root level processor that supports pipeline processors
- Update the
LoggerContextto support setting a new root processor.
The concrete problem: with today's LogRecordProcessor interface, a configuration like
logger_provider:
processors:
my_logs_filter:
my_log_record_sanitizer:
event_to_span_bridge:
batch:
exporter:
otlp_http:
fans out via MultiLogRecordProcessor instead of forming a pipeline: each processor gets an independent copy of the record via OnEmit(), which transfers ownership. That means N copies of the record (N times the recording cost) and the exporter at the end receives the raw, unfiltered/unsanitized record rather than the output of the upstream stages, since mutations made by an earlier stage are made on a copy that is then discarded.
See the full discussion: https://github.com/open-telemetry/opentelemetry-cpp/pull/4309#pullrequestreview-4953987049 and https://github.com/open-telemetry/opentelemetry-cpp/pull/4309#discussion_r3798294882
Scope
- Standardize SDK built-in exporters (OTLP, ostream) on a
ReadWriteLogRecord-style recordable. - New processor interface for composable pipeline processors that mutate data in place without owning the recordable lifecycle.
- New root-level processor implementing a pipeline of these composable processors.
LoggerContextsupport for installing this new root processor.- Once the above lands, implement
EventToSpanEventBridgeProcessoragainst the new interface and reintroduce it into declarative configuration (currently the config model/parser accepts theevent_to_span_bridgeprocessor block, butSdkBuilderonly logs a warning that it is not yet supported; see theFIXME-SDKcomment insdk_builder.cc).
Status
Pending design and implementation of the components above. #4309 was scoped down to just the configuration model and YAML parser per this discussion, with the actual processor implementation deferred until this issue is resolved.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing LogRecordProcessor and MultiLogRecordProcessor design, then inspect LoggerContext and the FIXME-SDK comment in sdk_builder.cc. Define the required ReadWriteLogRecord and composable pipeline components, root processor, and LoggerContext integration before implementing EventToSpanEventBridgeProcessor. Done means the architecture works without transferring record ownership and the bridge can be reintroduced into declarative configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100