[Filebeat]How should I configure Filebeat to correctly remove fields like 'agent' and 'host'?

**"How should I configure Filebeat to correctly remove fields like 'agent' and 'host'? I've set up the drop processor, but it doesn't seem to be working."**

this is my config

```
# ============================== Filebeat inputs ===============================
filebeat.inputs:
- type: filestream
id: docker-containers-logs
enabled: true
paths:
- /var/lib/docker/containers/*/*.log
scan_frequency: 30s
fields:
dataset: docker
env: sit-int

parsers:
- ndjson:
message_key: "log"
target: ""
add_error_key: true
overwrite_keys: true

processors:
- decode_json_fields:
fields: ["message"]
target: "log"
overwrite_keys: true
process_array: false
max_depth: 5

- timestamp:
field: "time"
target: "@timestamp"
# **Explicitly specify to overwrite the default @timestamp**
layouts:
- '2006-01-02T15:04:05.999999999Z'
timezone: "UTC"

- rename:
fields:
- from: "log.time"
to: "log.app_time"
fail_on_error: false

# **Add script processor: clean up empty string fields**
- script:
lang: javascript
id: drop_empty_fields
# **Check and clean up empty string values in header and body fields**
source: >
function process(event) {
var headerValue = event.Get("log.header");
var bodyValue = event.Get("log.body");
// **If log.header is an empty string, delete the field**
if (headerValue === '') {
event.Delete("log.header");
}
// **If log.body is an empty string, delete the field**
if (bodyValue === '') {
event.Delete("log.body");
}
}

- drop_fields:
fields: ["agent", "ecs", "host", "input", "stream", "message"]
ignore_missing: true
# ============================== Outputs ===============================
output.elasticsearch:
hosts: ["https://192.168.202.241:9200"]
username: "elastic"
password: "-NZ6+-EAH*97=DqKOcB*"
ssl:
certificate_authorities: ["/app/filebeat/filebeat-9.1.3-linux-x86_64/certs/http_ca.crt"]
indices:
- index: "logs-docker-%{[fields.env]}" # **Data stream name**
when.equals: # **Condition: when the field value equals the specified value**
fields.dataset: "docker" # **Check the fields.dataset field**
# **Performance tuning parameters**
bulk_max_size: 500
worker: 2
retry:
max_retries: 5
max_backoff: 5s

queue:
mem:
events: 8192
# **Default is 4096, can be doubled [2,4](@ref)**
flush.min_events: 2048

ilm:
enabled: false # **Disable ILM on the output side**
setup.ilm:
enabled: false # **Disable ILM on the setup side, key to preventing default behavior**
setup.template:
enabled: false # **Disable automatic template upload**
```

**The format of the collected logs is as follows:**
```
{
"_index": ".ds-logs-docker-sit-int-2025.11.16-000022",
"_id": "AZqRAN-HiZSPyJsNfyUM",
"_version": 1,
"_source": {
"@timestamp": "2025-11-17T08:48:10.501Z",
"agent": {
"type": "filebeat",
"version": "9.1.3",
"ephemeral_id": "3b05f246-5d7c-428d-88c8-e1de13878346",
"id": "52f73516-74a6-48c9-82f3-7cf3f70d5c22",
"name": "application001"
},
"fields": {
"dataset": "docker",
"env": "sit-int"
},
"log": {
"level": "INFO",
"app_time": "2025-11-17 08:48:10.501",
"name": "auth-center",
"message": "[3eaa93bb35bea5cc87e72b57869175bb-7b6ddbd39db0908a] do task: auth-center/service/user_token.(*checkUserTokenService).updateUserTokenTask-fm"
},
"ecs": {
"version": "8.0.0"
},
"time": "2025-11-17T08:48:10.501090494Z",
"host": {
"name": "application001"
}
},
"fields": {
"log.message": [
"[3eaa93bb35bea5cc87e72b57869175bb-7b6ddbd39db0908a] do task: auth-center/service/user_token.(*checkUserTokenService).updateUserTokenTask-fm"
],
"fields.env": [
"sit-int"
],
"log.name": [
"auth-center"
],
"agent.type": [
"filebeat"
],
"@timestamp": [
"2025-11-17T08:48:10.501Z"
],
"agent.id": [
"52f73516-74a6-48c9-82f3-7cf3f70d5c22"
],
"fields.dataset": [
"docker"
],
"ecs.version": [
"8.0.0"
],
"log.level": [
"INFO"
],
"agent.ephemeral_id": [
"3b05f246-5d7c-428d-88c8-e1de13878346"
],
"log.app_time": [
"2025-11-17T08:48:10.501Z"
],
"agent.version": [
"9.1.3"
],
"agent.name": [
"application001"
],
"host.name": [
"application001"
],
"time": [
"2025-11-17T08:48:10.501090494Z"
]
}
}
```

**As the same time,The configuration file I used for EMQX collection successfully removed the agentand hostfields.**
```
filebeat.inputs:
- type: filestream
enabled: true
id: emqx-logs-simple
paths:
- /app/akubela/middleware/emqx-dev/log/emqx.log*
# This path will match all log files including emqx.log, emqx.log.1, emqx.log.2.gz, etc.
fields:
name: 'emqx-dev'
env: 'sit-int'
fields_under_root: true
# Key configuration: Promote fields within the 'fields' section to the root level of the event

processors:
- dissect:
tokenizer: '%{time} %{message}'
field: "message"
target_prefix: ""
overwrite_keys: true

- drop_fields:
fields: ["agent", "ecs", "host", "input", "stream", "log"]
ignore_missing: true

# ============================== Outputs ===============================
output.elasticsearch:
hosts: ["https://192.168.202.241:9200"]
username: "elastic"
password: "-NZ6+-EAH*97=DqKOcB*"
ssl:
certificate_authorities: ["/app/filebeat/filebeat-9.1.3-linux-x86_64/certs/http_ca.crt"]
indices:
- index: "emqx-logs-%{[name]}-%{[env]}" # Data stream name

# Performance tuning parameters
bulk_max_size: 500
worker: 2
retry:
max_retries: 5
max_backoff: 5s
```

Start by reproducing the shown Filebeat filestream configuration with the sample Docker event and inspect the resulting event after processors run. Compare it with the working EMQX configuration, then verify that the requested fields are absent from the emitted event while the remaining log fields and index selection still work.

Written by the indexing model from the issue text.