elastic / elastic/elastic-agent

Duplicate data collection between generic inputs and specific integrations

Open
#274 6 comments 4 reactions 0 assignees View on GitHub
discuss
Dominant language
Go
Stars
275
Forks
264
Avg merge
1d 20h
Merged PRs (30d)
298

Description

Today, we have generic inputs like `Container Logs` or `Custom Logs`. Based on dynamic variables and various providers we can also set this generic inputs to collect logs from all of the log files in a specific path.
For example in Kubernetes (or similarly in Docker) we can set the path like `/var/log/containers/*${kubernetes.container.id}.log` and we will have the input to be set for all the containers based on their `container.id` value.

In addition we can specify
```
- name: nginx
type: nginx/logs
use_output: default
data_stream:
namespace: default
streams:
- data_stream:
dataset: nginx.access
type: logs
paths:
- '/var/log/containers/*${kubernetes.container.id}.log'
condition: ${kubernetes.labels.app} == 'nginx'
```
which is an input based on a condition for `nginx`.
In the end we will have this Pod's logs to be collected twice, from the generic `Container Logs` input and one for the specific one for `Nginx`.

cc: @mukeshelastic

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.