elastic / elastic/apm-data

Hostname log field overwritten or even cleared when k8s attribute is present

Open
#386 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
12
Forks
31
Avg merge
8h 55m
Merged PRs (30d)
2

Description

I came to this while processing OTEL log messages. When `host.name` OTEL attribute is present in the resource attributes, it is converted to `Host.Hostname` field:

https://github.com/elastic/apm-data/blob/46f8e38486b772090fbf1fdccc8eeea13c513075/input/otlp/metadata.go#L173-L177

That is fine. But when there is any k8s attribute present, it is either rewritten, or even cleared in post-processing phase:

https://github.com/elastic/apm-data/blob/46f8e38486b772090fbf1fdccc8eeea13c513075/model/modelprocessor/hostname.go#L41-L53

The code is there for years (previously in apm-server repo), so I believe this was done to fill some information APM agent did not have.

I discovered this by setting `k8s.pod.name` from `host.name` attribute in OTEL Collector to fill some missing information manually and suddenly the _smart_ `resource` column in Kibana Logs stopped showing any host name. This looks like a bug to me.

I also discovered that the value of OTEL attribute `host.name` has no influence on Kibana Logs `resource` column display and it always shows `k8s.node.name` when present. For us more valuable is to actually show `k8s.pod.name`, so we know the originating instance, not the Kubernetes Node with many PODs. Showing the Node name serving many PODs does not make much sense for us actually.

I think it would make sense to keep the `Host.Hostname` value as-is in case it is non-empty. So no overwriting, no clearing.

![Image](https://github.com/user-attachments/assets/a4d6cf11-2ae0-45bd-ae3a-14d2eee98453)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.