elastic / elastic/logstash

Configuration of the Metric

Open
#4,501 5 comments 0 reactions 0 assignees View on GitHub
discuss monitoring v5.5.0
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

The current way for logstash to send metrics to an external services is by using an internal pipeline.
This pipeline need to be configured the appropriate way for the branch https://github.com/elastic/logstash/tree/feature/metrics to correctly work.

This pipeline has the current format:

```
input { metrics {} } # required to send events down the queues.
output {
abc {
host => "127.0.0.1"
user => "abc"
password => "hello world"
}
}
```

---
## Possible solutions
### Solution 1: settings.yml

My initial idea was to use the newly introduced `settings.yml` (https://github.com/elastic/logstash/pull/4499) file to configure the pipeline and also hide the implementation details of a real pipeline doing the work, if we decide to go with something else than a pipeline we can change the implementation.

```
metrics.shippers.abc: "127.0.0.1"
metrics.shippers.user: "127.0.0.1"
metrics.shippers.password: "127.0.0.1"
```
### Solution 2. Adding a `metric.conf`

Another solution is to add a `metric.conf` as a template and use the same syntax as normal configuration files, this configuration will contains the `metrics` in the input.

```
input { metrics {} }
output {
abc {
host => "127.0.0.1"
user => "abc"
password => "hello world"
}
}
```
### Solution 3: A Metric bock

Add a metric block inside the current logstash syntax, this was the original idea at the beginning of the development, so a normal logstash configuration would look like this:

```
input { generator {} }
output { elasticsearch {} }
metrics {
abc {
host => "127.0.0.1"
user => "abc"
password => "hello world"
}
}
```

I dont really like the 3rd solution because it require changes in the AST, but it might be better in the context of clustering.
### Note:

When we are configuring the metrics its more than just defining a shipper, we can configure the web api too (SSL, Port) and the Snapshot resolution. All of theses options can be easily defined in the yml files.

@jsvd @andrewvc @purbon and @colinsurprenant what are your thoughts on this?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.