elastic / elastic/integrations
[Nginx Ingress Controller]: `filestream input ID 'filestream-nginx_ingress_controller.access-XXX' is duplicated: input will NOT start
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
### Integration Name
Nginx Ingress Controller Logs [nginx_ingress_controller]
### Dataset Name
_No response_
### Integration Version
v1.12.0
### Agent Version
9.2.1
### Agent Output Type
elasticsearch
### Elasticsearch Version
9.3.0-SNAPSHOT
### OS Version and Architecture
Kubernetes
### Software/API Version
_No response_
### Error Message
Nov 11, 2025 @ 12:21:49.237 error Error creating runner from config: ErrNonReloadable: filestream input with ID 'filestream-nginx_ingress_controller.error-71bdf59b-6173-4a0b-9456-9501d0c5859d' already exists, this will lead to data duplication, please use a different ID
Nov 11, 2025 @ 12:21:49.237 error filestream input ID 'filestream-nginx_ingress_controller.error-71bdf59b-6173-4a0b-9456-9501d0c5859d' is duplicated: input will NOT start
Nov 11, 2025 @ 12:21:49.237 error Error creating runner from config: ErrNonReloadable: filestream input with ID 'filestream-nginx_ingress_controller.access-71bdf59b-6173-4a0b-9456-9501d0c5859d' already exists, this will lead to data duplication, please use a different ID
Nov 11, 2025 @ 12:21:49.237 error filestream input ID 'filestream-nginx_ingress_controller.access-71bdf59b-6173-4a0b-9456-9501d0c5859d' is duplicated: input will NOT start
### Event Original
_No response_
### What did you do?
1. Create a new Kubernets cluster (using Kind)
2. Install the Nginx Ingress Controller by running the [quick start](https://kubernetes.github.io/ingress-nginx/deploy/#quick-start) helm command
3. Edited the number of replicas on the deployment to 2 (there needs to be more than one Nginx ingress controller container running)
4. Installed the integration accepting all the defaults and making sure error and access logs were enabled
5. Deployed the Elastic Agent following the instructions on Fleet UI
### What did you see?
* The error message mentioned above
* The Elastic Agent goes unhealthy after a while:
### What did you expect to see?
No errors, the Elastic Agent remains healthy
### Anything else?
The root cause of the problem is the fact that the integration does not define an unique Filestream ID per input started.
The integration sets the default paths to [`/var/log/containers/*${kubernetes.container.id}.log`](https://github.com/elastic/integrations/blob/8769c4f83df10fc8d2b20ce17266040bd1fd229f/packages/nginx_ingress_controller/data_stream/access/manifest.yml#L13), which starts one Filestream input per container. However there is no ID defined in the [`integration configuraiton`](https://github.com/elastic/integrations/blob/main/packages/nginx_ingress_controller/data_stream/access/agent/stream/stream.yml.hbs), so every input is started with the same ID, this causes one of two problems:
- For Elastic Agent versions < 9.0.0, all inputs are started but data is duplicated when Filebeat (or the inputs) are restarted
- For Elastic Agent >= 9.0.0, only the first Filestream input is started, therefore not all logs are collected.
Contributor guide
Assessment
This issue has not been assessed yet.