elastic / elastic/docs-content
[Beats Inputs] Pipeline attribute in Beats inputs behavior and special handling
- Dominant language
- No language data
- Stars
- 47
- Forks
- 261
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 116
Description
Document the `pipeline` attribute in the inputs, such as:
- https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-log#_pipeline_16
- https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-filestream#_pipeline_9
- https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-udp#_pipeline_26
- TCP...
- All inputs exposing `pipeline`...
Will end up in a field called `@metadata.pipeline` when using Kafka & Logstash outputs.
In order to Elasticsearch to "execute" the pipeline, it will require special handling by the Logstash or anything consuming from Kafka to read `@metadata.pipeline` and pass it to the ES output.
E.g. for Logstash
```
if [@metadata][pipeline] {
elasticsearch {
hosts => ["http://localhost:9200"]
# cloud_id => "..."
data_stream => "true"
api_key => ""
data_stream => true
ssl_enabled => true
ssl_certificate_authorities => ""
pipeline => "%{[@metadata][pipeline]}" # <-------------------------------
}
} else {
elasticsearch {
hosts => ["http://localhost:9200"]
# cloud_id => "..."
data_stream => "true"
api_key => ""
data_stream => true
ssl_enabled => true
ssl_certificate_authorities => ""
}
}
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.