redpanda-data / redpanda-data/benthos
Streams mode: spans carry no stream identity
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 571
- Forks
- 120
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 18
Description
In streams mode every span is named after the component type (input_generate, mapping, output_sql_raw), so spans from different streams look identical in a tracing backend. There is no config-level way to tell them apart: label is not applied to spans (internal/component/input/async_reader.go, processor/auto_observed.go, output/async_writer.go build the name from the type only), and the tracer's tags are resource attributes shared by the whole process.
Logs and metrics already have this: Manager.forStream adds a stream field to the logger and a stream label to the metrics, but passes the tracer provider through untouched. The tracer is the odd one out.
Use case: one Redpanda Connect process running ~30 streams in streams mode, exporting to Tempo via OTLP. I want to write { span.stream = "knx" } in TraceQL, or filter on the stream in Jaeger. Today the only way is a group_by_value processor in each stream, which happens to set a group attribute on its span; that works, but it is a batching processor used for a side effect, and the attribute lands on a child span rather than the root.
Proposal: forStream wraps the tracer provider so that every span it starts carries stream=<id>, added as a span start option so samplers can see it. No config change, applies to all streams at once, and it mirrors what logs and metrics already do. PR to follow.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Manager.forStream and the span-name construction in internal/component/input/async_reader.go, processor/auto_observed.go, and output/async_writer.go. Trace how the provider reaches each stream, then verify that spans receive the stream identity at start and that the attribute is available for sampling and TraceQL or Jaeger filtering without a config change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100