Implement processor for journald to opentelemetry semantic mapping
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 58
Description
**Is your feature request related to a problem? Please describe.**
Journald implements structured fields with certain semantic. Opentelemetry has their semantic convention with similar fields.
https://github.com/open-telemetry/opentelemetry-specification/pull/4995 tries to describe the mapping between these in the opentelemetry appendix.
**Describe the solution you'd like**
I would like to propose a built in processor that would convert from a jorunald input (systemd) to an opentelemetry spec compliant envelope with the semantic of the fields preserved.
**Describe alternatives you've considered**
- I tried to implement the mapping by combining the existing processorts (rename, etc.) but they don't allow all the required operataions-
- I also tried to implement this as lua and WASM plugin. This works but it adds additional dependencies to fluentbit.
- Last I also implemented it as a processor in C, this works, see https://github.com/bachp/fluent-bit/tree/journald-mapping
**Additional context**
- Currently I have an out of tree version of this plugin that I can keep using if this is not desired by upstream.
- **The main question to answer is if this processor is something that should live upstream in fluent-bit or external.**
Contributor guide
Research direction
Start by reviewing the out-of-tree C implementation at https://github.com/bachp/fluent-bit/tree/journald-mapping and the OpenTelemetry specification PR #4995 to understand the proposed field mapping. The issue is done when maintainers decide whether this belongs upstream or externally; if accepted, the processor should preserve journald semantics in an OpenTelemetry-compliant envelope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, linux
- Domain
- observability-sre, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100