fluent / fluent/fluentd-docs-gitbook
Elasticsearch & Opensearch output plugins have unexpected/undocumented behavior
- Dominant language
- CSS
- Stars
- 46
- Forks
- 136
- Avg merge
- 4h 47m
- Merged PRs (30d)
- 16
Description
### Describe the bug
It took me hours to debug this issue, and it wasn't until digging into the plugin's README files that I figured out what is going on!
Both the Opensearch & Elasticsearch plugins mention the logstash_format parameter impacting the index. What it does *not* say is that if you set logstash_format = false, it *also* sets include_timestamp=false. This results in records not being usable in Opensearch Dashboards & Kibana, as there are no time records.
### Link to the problematic documentation
https://docs.fluentd.org/output/opensearch#logstash_format-optional
https://docs.fluentd.org/output/elasticsearch#logstash_format-optional
### Expected explanation
If true, Fluentd uses the conventional index name format logstash-%Y.%m.%d (default: false). This option supersedes the `index_name` option.
In addition, `include_timestamp` (default: false) is set to true, which sends timestamp information which can be used by (Opensearch Dashboards / Kibana).
### Additional context
The documentation here: https://github.com/fluent/fluent-plugin-opensearch#include_timestamp does indicate this behavior.
Note that the documentation here is much more thorough and authoritative overall.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.