elastic / elastic/beats

add_fields processor: does not overwrite the target field if it already exists

Open
#40,780 5 comments 0 reactions 0 assignees View on GitHub
Stalled Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

I am using `add_fields` processor to add `orchestrator.cluster.name`
```
- add_fields:
target: orchestrator.cluster
fields:
name: "test"
```
Note: it is a way we mention in our public doc - https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-kubernetes.html#_dashboard_32 for cases when cluster.name can't be detected automatically.

When testing (on kind cluster, where orchestrator.cluster.name can be defined) I've noticed that the field value is added to the existing field, but not get overwrite as claimed in [doc](https://www.elastic.co/guide/en/beats/metricbeat/current/add-fields.html) and suppose to be in [tests](https://github.com/elastic/beats/blob/v8.15.1/libbeat/processors/actions/add_fields_test.go#L45-L51):

![Image](https://github.com/user-attachments/assets/5de7e479-aaf3-4ea3-8024-e561c8cefa3e)

8.16:

```
{
"_index": ".ds-metricbeat-8.16.0-2024.09.11-000001",
"_id": "40if4ZEBJGFURqBQZ8MC",
"_version": 1,
"_score": 0,
"_source": {
"@timestamp": "2024-09-11T15:06:45.113Z",
"metricset": {
"name": "node",
"period": 10000
},
"kubernetes": {
"node": {
"memory": {
"usage": {
"bytes": 4808962048
},
"workingset": {
"bytes": 2266112000
},
"rss": {
"bytes": 1402314752
},
"pagefaults": 57223252,
"majorpagefaults": 1610,
"available": {
"bytes": 14503260160
}
},
"network": {
"tx": {
"bytes": 596430470,
"errors": 0
},
"rx": {
"bytes": 1133629283,
"errors": 0
}
},
"start_time": "2024-08-26T07:48:42Z",
"name": "kind-control-plane",
"fs": {
"capacity": {
"bytes": 369759891456
},
"used": {
"bytes": 219972317184
},
"inodes": {
"used": 3508210,
"free": 19445774,
"count": 22953984
},
"available": {
"bytes": 130970845184
}
},
"runtime": {
"imagefs": {
"available": {
"bytes": 130970845184
},
"capacity": {
"bytes": 369759891456
},
"used": {
"bytes": 3731861504
}
}
},
"cpu": {
"usage": {
"nanocores": 229101086,
"core": {
"ns": 49213133124000
}
}
}
},
"labels": {
"beta_kubernetes_io/os": "linux",
"kubernetes_io/arch": "amd64",
"kubernetes_io/hostname": "kind-control-plane",
"kubernetes_io/os": "linux",
"node-role_kubernetes_io/control-plane": "",
"beta_kubernetes_io/arch": "amd64"
}
},
"agent": {
"ephemeral_id": "cbe5bf1e-f613-4c77-9517-530e189263f5",
"id": "c6bad5b7-9f96-4701-812e-bd08be0e0d39",
"name": "kind-control-plane",
"type": "metricbeat",
"version": "8.16.0"
},
"orchestrator.cluster": {
"name": "test"
},
"service": {
"address": "https://kind-control-plane:10250/stats/summary",
"type": "kubernetes"
},
"event": {
"dataset": "kubernetes.node",
"module": "kubernetes",
"duration": 554803
},
"host": {
"name": "kind-control-plane"
},
"orchestrator": {
"cluster": {
"url": "kind-control-plane:6443",
"name": "kind"
}
},
"ecs": {
"version": "8.0.0"
}
},
"fields": {
"orchestrator.cluster.name": [
"kind",
"test"
],
"kubernetes.labels.kubernetes_io/arch": [
"amd64"
],
"kubernetes.node.runtime.imagefs.available.bytes": [
130970845184
],
"service.type": [
"kubernetes"
],
"kubernetes.labels.beta_kubernetes_io/arch": [
"amd64"
],
"kubernetes.labels.kubernetes_io/hostname": [
"kind-control-plane"
],
"agent.name": [
"kind-control-plane"
],
"host.name": [
"kind-control-plane"
],
"beats_state.state.host.name": [
"kind-control-plane"
],
"kubernetes.node.runtime.imagefs.capacity.bytes": [
369759891456
],
"agent.hostname": [
"kind-control-plane"
],
"agent.id": [
"c6bad5b7-9f96-4701-812e-bd08be0e0d39"
],
"ecs.version": [
"8.0.0"
],
"service.address": [
"https://kind-control-plane:10250/stats/summary"
],
"agent.version": [
"8.16.0"
],
"kubernetes.node.fs.inodes.used": [
3508210
],
"kubernetes.node.network.rx.bytes": [
1133629283
],
"kubernetes.node.network.tx.bytes": [
596430470
],
"kubernetes.node.fs.inodes.count": [
22953984
],
"kubernetes.node.memory.majorpagefaults": [
1610
],
"kubernetes.node.fs.inodes.free": [
19445774
],
"kubernetes.node.name": [
"kind-control-plane"
],
"kubernetes.node.runtime.imagefs.used.bytes": [
3731861504
],
"kubernetes.labels.beta_kubernetes_io/os": [
"linux"
],
"kubernetes.labels.node-role_kubernetes_io/control-plane": [
""
],
"kubernetes.labels.kubernetes_io/os": [
"linux"
],
"kubernetes.node.memory.pagefaults": [
57223252
],
"kubernetes.node.start_time": [
"2024-08-26T07:48:42.000Z"
],
"kubernetes.node.fs.used.bytes": [
219972317184
],
"kubernetes.node.memory.rss.bytes": [
1402314752
],
"agent.type": [
"metricbeat"
],
"kubernetes.node.cpu.usage.core.ns": [
49213133124000
],
"orchestrator.cluster.url": [
"kind-control-plane:6443"
],
"logstash_stats.timestamp": [
"2024-09-11T15:06:45.113Z"
],
"event.module": [
"kubernetes"
],
"kubernetes.node.cpu.usage.nanocores": [
229101086
],
"beats_state.timestamp": [
"2024-09-11T15:06:45.113Z"
],
"timestamp": [
"2024-09-11T15:06:45.113Z"
],
"kubernetes.node.network.tx.errors": [
0
],
"kubernetes.node.memory.usage.bytes": [
4808962048
],
"kibana_stats.timestamp": [
"2024-09-11T15:06:45.113Z"
],
"kubernetes.node.fs.available.bytes": [
130970845184
],
"metricset.period": [
10000
],
"kubernetes.node.network.rx.errors": [
0
],
"kubernetes.node.memory.available.bytes": [
14503260160
],
"event.duration": [
554803
],
"metricset.name": [
"node"
],
"kubernetes.node.fs.capacity.bytes": [
369759891456
],
"kubernetes.node.memory.workingset.bytes": [
2266112000
],
"@timestamp": [
"2024-09-11T15:06:45.113Z"
],
"agent.ephemeral_id": [
"cbe5bf1e-f613-4c77-9517-530e189263f5"
],
"event.dataset": [
"kubernetes.node"
]
}
}
```

specifically notice here:
```
"_source": {
...
"orchestrator.cluster": {
"name": "test"
},
...
"orchestrator": {
"cluster": {
"url": "kind-control-plane:6443",
"name": "kind"
}
},
...
},
"fields": {
"orchestrator.cluster.name": [
"kind",
"test"
],
```

That behavior might have changed after this PR: https://github.com/elastic/beats/pull/14407

```
{
"_index": "metricbeat-7.5.2-2024.09.11-000001",
"_id": "m1B15ZEBJGFURqBQ7sbZ",
"_version": 1,
"_score": 0,
"_source": {
"@timestamp": "2024-09-12T08:59:55.997Z",
"service": {
"address": "kind-control-plane:10250",
"type": "kubernetes"
},
"kubernetes": {
"pod": {
"name": "elastic-agent-llh2m",
"uid": "47e3d2a2-c3c9-4fad-9d36-ad74ed7f0548"
},
"labels": {
"controller-revision-hash": "6fd88bf58b",
"pod-template-generation": "10",
"app": "elastic-agent"
},
"container": {
"memory": {
"usage": {
"limit": {
"pct": 0.03879597028395844
},
"bytes": 650584064,
"node": {
"pct": 0.03879597028395844
}
},
"workingset": {
"bytes": 650584064
},
"rss": {
"bytes": 617177088
},
"pagefaults": 18396612,
"majorpagefaults": 33,
"available": {
"bytes": 83419136
}
},
"rootfs": {
"available": {
"bytes": 128244932608
},
"capacity": {
"bytes": 369759891456
},
"used": {
"bytes": 62169088
},
"inodes": {
"used": 69
}
},
"logs": {
"available": {
"bytes": 128244932608
},
"capacity": {
"bytes": 369759891456
},
"used": {
"bytes": 13045760
},
"inodes": {
"used": 5,
"free": 19422833,
"count": 22953984
}
},
"start_time": "2024-09-10T16:03:45Z",
"name": "elastic-agent",
"cpu": {
"usage": {
"nanocores": 19947667,
"core": {
"ns": 2722178877000
},
"node": {
"pct": 0.0012467291875
},
"limit": {
"pct": 0.0012467291875
}
}
}
},
"namespace": "kube-system",
"node": {
"name": "kind-control-plane"
}
},
"event": {
"dataset": "kubernetes.container",
"module": "kubernetes",
"duration": 32475075
},
"agent": {
"hostname": "kind-control-plane",
"id": "c6bad5b7-9f96-4701-812e-bd08be0e0d39",
"version": "7.5.2",
"type": "metricbeat",
"ephemeral_id": "a6b39abc-f565-4597-a80c-d5434f13be9d"
},
"ecs": {
"version": "1.1.0"
},
"host": {
"name": "kind-control-plane"
},
"orchestrator.cluster": {
"name": "test"
},
"metricset": {
"period": 10000,
"name": "container"
}
},
"fields": {
"kubernetes.node.name": [
"kind-control-plane"
],
"orchestrator.cluster.name": [
"test"
],
"kubernetes.container.logs.inodes.free": [
19422833
],
"kubernetes.container.logs.inodes.used": [
5
],
"kubernetes.pod.uid": [
"47e3d2a2-c3c9-4fad-9d36-ad74ed7f0548"
],
"kubernetes.container.logs.used.bytes": [
13045760
],
"kubernetes.container.memory.usage.bytes": [
650584064
],
"kubernetes.container.rootfs.used.bytes": [
62169088
],
"kubernetes.container.start_time": [
"2024-09-10T16:03:45.000Z"
],
"kubernetes.container.cpu.usage.nanocores": [
19947667
],
"service.type": [
"kubernetes"
],
"agent.type": [
"metricbeat"
],
"kubernetes.container.cpu.usage.core.ns": [
2722178877000
],
"event.module": [
"kubernetes"
],
"kubernetes.labels.app": [
"elastic-agent"
],
"kubernetes.container.rootfs.inodes.used": [
69
],
"kubernetes.namespace": [
"kube-system"
],
"kubernetes.container.memory.usage.limit.pct": [
0.039
],
"kubernetes.pod.name": [
"elastic-agent-llh2m"
],
"host.name": [
"kind-control-plane"
],
"kubernetes.container.memory.usage.node.pct": [
0.039
],
"kubernetes.container.memory.majorpagefaults": [
33
],
"kubernetes.container.name": [
"elastic-agent"
],
"kubernetes.container.logs.available.bytes": [
128244932608
],
"kubernetes.container.logs.inodes.count": [
22953984
],
"kubernetes.container.memory.available.bytes": [
83419136
],
"metricset.period": [
10000
],
"kubernetes.container.cpu.usage.node.pct": [
0.001
],
"kubernetes.container.memory.pagefaults": [
18396612
],
"kubernetes.container.rootfs.capacity.bytes": [
369759891456
],
"agent.hostname": [
"kind-control-plane"
],
"kubernetes.labels.pod-template-generation": [
"10"
],
"event.duration": [
32475075
],
"metricset.name": [
"container"
],
"kubernetes.container.cpu.usage.limit.pct": [
0.001
],
"kubernetes.container.logs.capacity.bytes": [
369759891456
],
"@timestamp": [
"2024-09-12T08:59:55.997Z"
],
"agent.id": [
"c6bad5b7-9f96-4701-812e-bd08be0e0d39"
],
"ecs.version": [
"1.1.0"
],
"kubernetes.labels.controller-revision-hash": [
"6fd88bf58b"
],
"service.address": [
"kind-control-plane:10250"
],
"kubernetes.container.rootfs.available.bytes": [
128244932608
],
"agent.ephemeral_id": [
"a6b39abc-f565-4597-a80c-d5434f13be9d"
],
"kubernetes.container.memory.rss.bytes": [
617177088
],
"agent.version": [
"7.5.2"
],
"kubernetes.container.memory.workingset.bytes": [
650584064
],
"event.dataset": [
"kubernetes.container"
]
}
}
```

For confirmed bugs, please report:
- Version: > 7.5.2
- Operating System:
- Discuss Forum URL:
- Steps to Reproduce:

Contributor guide

Open the contributing guide

Research direction

Start with libbeat/processors/actions/add_fields_test.go, especially the referenced lines, and compare its expectations with the add_fields behavior described in the issue. Review the linked add_fields documentation and PR 14407 for the claimed overwrite semantics. Done means an existing target field is overwritten as documented and the relevant test covers the observed case.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.