fluent / fluent/fluent-plugin-opensearch
security_exception causes data to silently drop
- Dominant language
- Ruby
- Stars
- 64
- Forks
- 23
- Avg merge
- 1h 53m
- Merged PRs (30d)
- 3
Description
(check apply)
- [x] read [the contribution guideline](https://github.com/fluent/fluent-plugin-opensearch/blob/master/CONTRIBUTING.md)
- [ ] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
#### Steps to replicate
When writing to an index the plugin does have permission to, the data writes correctly to OpenSearch.
When trying to write to an index the plugin does not have permission to, purposefully causing a security_exception, data appears to silently drop.
```
workers 4
root_dir /var/log/fluentd
@type http
port 8080
bind 0.0.0.0
XXXFILTERSXXX
@id out_opensearch
@type opensearch
@log_level debug
scheme https
ssl_verify true
host XXXHOSTXXX
port XXXPORTXXX
user XXXUSERXXX
password XXXPASSWORDXXX
target_index_key @target_index
remove_keys @target_index
compression_level default_compression
log_os_400_reason true
flush_interval 1s
retry_type periodic
retry_forever true
retry_wait 5s
@type stdout
@type stdout
```
Only messages seen output to stdout:
2022-03-17 14:11:31 +0000 [debug]: #0 [out_opensearch] Indexed (op = index), 6 security_exception
2022-03-17 14:11:31.901853800 +0000 fluent.debug: {"message":"[out_opensearch] Indexed (op = index), 6 security_exception"}
#### Expected Behavior or What you need to ask
Messages not uploaded due to security_exception should make their way to ERROR.
If not ERROR, perhaps a configurable label and optional re-tag?
#### Using Fluentd and OpenSearch plugin versions
OS version: Docker image built off ghcr.io/calyptia/fluentd:v1.14.5-debian-1.0
Bare Metal or within Docker or Kubernetes or others?: Docker image built off ghcr.io/calyptia/fluentd:v1.14.5-debian-1.0
Fluentd v1.0 or later: 1.14.5
OpenSearch plugin version: fluent-plugin-opensearch version 1.0.2
Contributor guide
Assessment
This issue has not been assessed yet.