fluent / fluent/fluentd-docs-gitbook

Elasticsearch & Opensearch output plugins have unexpected/undocumented behavior

Open
#419 1 comment 1 reaction 0 assignees View on GitHub
good first issue
Dominant language
CSS
Stars
46
Forks
136
Avg merge
4h 47m
Merged PRs (30d)
16

Description

### Describe the bug

It took me hours to debug this issue, and it wasn't until digging into the plugin's README files that I figured out what is going on!

Both the Opensearch & Elasticsearch plugins mention the logstash_format parameter impacting the index. What it does *not* say is that if you set logstash_format = false, it *also* sets include_timestamp=false. This results in records not being usable in Opensearch Dashboards & Kibana, as there are no time records.

### Link to the problematic documentation

https://docs.fluentd.org/output/opensearch#logstash_format-optional
https://docs.fluentd.org/output/elasticsearch#logstash_format-optional

### Expected explanation

If true, Fluentd uses the conventional index name format logstash-%Y.%m.%d (default: false). This option supersedes the `index_name` option.
In addition, `include_timestamp` (default: false) is set to true, which sends timestamp information which can be used by (Opensearch Dashboards / Kibana).

### Additional context

The documentation here: https://github.com/fluent/fluent-plugin-opensearch#include_timestamp does indicate this behavior.

Note that the documentation here is much more thorough and authoritative overall.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.