elastic / elastic/integrations

[Nginx Ingress Controller]: `filestream input ID 'filestream-nginx_ingress_controller.access-XXX' is duplicated: input will NOT start

Open
#15,941 1 comment 0 reactions 0 assignees View on GitHub
Integration:nginx_ingress_controller needs:triage Team:obs-ds-hosted-services
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

### Integration Name

Nginx Ingress Controller Logs [nginx_ingress_controller]

### Dataset Name

_No response_

### Integration Version

v1.12.0

### Agent Version

9.2.1

### Agent Output Type

elasticsearch

### Elasticsearch Version

9.3.0-SNAPSHOT

### OS Version and Architecture

Kubernetes

### Software/API Version

_No response_

### Error Message

Nov 11, 2025 @ 12:21:49.237 error Error creating runner from config: ErrNonReloadable: filestream input with ID 'filestream-nginx_ingress_controller.error-71bdf59b-6173-4a0b-9456-9501d0c5859d' already exists, this will lead to data duplication, please use a different ID
Nov 11, 2025 @ 12:21:49.237 error filestream input ID 'filestream-nginx_ingress_controller.error-71bdf59b-6173-4a0b-9456-9501d0c5859d' is duplicated: input will NOT start
Nov 11, 2025 @ 12:21:49.237 error Error creating runner from config: ErrNonReloadable: filestream input with ID 'filestream-nginx_ingress_controller.access-71bdf59b-6173-4a0b-9456-9501d0c5859d' already exists, this will lead to data duplication, please use a different ID
Nov 11, 2025 @ 12:21:49.237 error filestream input ID 'filestream-nginx_ingress_controller.access-71bdf59b-6173-4a0b-9456-9501d0c5859d' is duplicated: input will NOT start

### Event Original

_No response_

### What did you do?

1. Create a new Kubernets cluster (using Kind)
2. Install the Nginx Ingress Controller by running the [quick start](https://kubernetes.github.io/ingress-nginx/deploy/#quick-start) helm command
3. Edited the number of replicas on the deployment to 2 (there needs to be more than one Nginx ingress controller container running)
4. Installed the integration accepting all the defaults and making sure error and access logs were enabled
5. Deployed the Elastic Agent following the instructions on Fleet UI

### What did you see?

* The error message mentioned above
* The Elastic Agent goes unhealthy after a while:
Image

### What did you expect to see?

No errors, the Elastic Agent remains healthy

### Anything else?

The root cause of the problem is the fact that the integration does not define an unique Filestream ID per input started.

The integration sets the default paths to [`/var/log/containers/*${kubernetes.container.id}.log`](https://github.com/elastic/integrations/blob/8769c4f83df10fc8d2b20ce17266040bd1fd229f/packages/nginx_ingress_controller/data_stream/access/manifest.yml#L13), which starts one Filestream input per container. However there is no ID defined in the [`integration configuraiton`](https://github.com/elastic/integrations/blob/main/packages/nginx_ingress_controller/data_stream/access/agent/stream/stream.yml.hbs), so every input is started with the same ID, this causes one of two problems:
- For Elastic Agent versions < 9.0.0, all inputs are started but data is duplicated when Filebeat (or the inputs) are restarted
- For Elastic Agent >= 9.0.0, only the first Filestream input is started, therefore not all logs are collected.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.