elastic / elastic/beats

[beatreceiver] Remove ES exporter specific attributes present in otelconsumer with kafka/logstash output

Open
#48,549 4 comments 0 reactions 0 assignees View on GitHub
Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

Currently, we add ES exporter specific attributes to otelconsumer.
1. `elasticsearch.ingest_pipeline` in log records [code](https://github.com/elastic/beats/blob/main/x-pack/libbeat/outputs/otelconsumer/otelconsumer.go#L129-L133)
2. And `elastic.mapping.mode` in scope attributes [code](https://github.com/elastic/beats/blob/main/x-pack/libbeat/outputs/otelconsumer/otelconsumer.go#L93)

They must be stripped when using kafka/logstash output as these are not required in the context of non-elasticsearch backend.

For more context see https://github.com/elastic/beats/pull/48494#discussion_r2717559221

Contributor guide

Open the contributing guide

Research direction

Start in x-pack/libbeat/outputs/otelconsumer/otelconsumer.go at the referenced lines 93 and 129-133, and review the discussion in PR 48494. Trace how the Kafka and Logstash output paths handle scope and log-record attributes, then verify that elastic.mapping.mode and elasticsearch.ingest_pipeline are stripped there while the issue’s non-Elasticsearch behavior is satisfied.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.