fluent / fluent/fluent-plugin-opensearch
Insufficient Permission to Create Index is a Debug Level Log
- Dominant language
- Ruby
- Stars
- 64
- Forks
- 23
- Avg merge
- 1h 53m
- Merged PRs (30d)
- 3
Description
(check apply)
- [ x ] read [the contribution guideline](https://github.com/fluent/fluent-plugin-opensearch/blob/master/CONTRIBUTING.md)
- [ ] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
#### Steps to replicate
Fluentd Config:
```
Log_Level debug
@type http
port 9880
bind 0.0.0.0
body_size_limit 32m
keepalive_timeout 10s
@type opensearch
@log_level debug
with_transporter_log true
host 192.168.122.165
port 9200
scheme https
ssl_verify true
ssl_version TLSv1_2
ca_file /opt/certs/opensearch-ca.pem
client_cert /opt/certs/opensearch-fluentd.pem
client_key /opt/certs/opensearch-fluentd.key
index_name fluentd-test
```
Opensearch Permission: The "indices:admin/create" permission is missing from the opensearch index on the user's role, preventing the opensearch user fluentd uses from creating indices.
Message:
```
2022-10-16 15:09:48 +0000 [debug]: #0 < {"took":4,"errors":true,"items":[{"index":{"_index":"fluentd-test","_id":null,"status":403,"error":{"type":"security_exception","reason":"no permissions for [indices:admin/create] and User [name=fluentd, backend_roles=[], requestedTenant=null]"}}},
```
#### Expected Behavior or What you need to ask
I expected the error to appear even when logs are set to info.
Instead, everything appeared to be silently working (given fluentd's feedback), but the log indices were not getting created in opensearch.
The problem only became apparent once I set the logs to debug.
#### Using Fluentd and OpenSearch plugin versions
* OS version: Ubuntu Focal
* Bare Metal or within Docker or Kubernetes or others: Opensearch in kvm, fluentd in docker
* Fluentd v1.0 or later:
* paste result of ``fluentd --version`` or ``td-agent --version``: `fluentd 1.14.0`
* OpenSearch plugin version
* paste boot log of fluentd or td-agent: `gem 'fluent-plugin-opensearch' version '1.0.8'`
* paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock:
```
*** LOCAL GEMS ***
async (1.30.1)
async-http (0.54.0)
async-io (1.33.0)
async-pool (0.3.9)
aws-eventstream (1.2.0)
aws-partitions (1.646.0)
aws-sdk-core (3.160.0)
aws-sigv4 (1.5.2)
bigdecimal (default: 1.4.1)
bundler (default: 1.17.2)
cmath (default: 1.0.0)
concurrent-ruby (1.1.10)
console (1.15.0)
cool.io (1.7.1)
csv (default: 3.0.9)
date (default: 2.0.3)
dbm (default: 1.0.0)
did_you_mean (1.3.0)
e2mmap (default: 0.1.0)
etc (default: 1.0.1)
excon (0.93.0)
ext_monitor (0.1.2)
faraday (1.10.2)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.0)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.4)
faraday-net_http (1.0.1)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.3)
faraday_middleware-aws-sigv4 (0.6.1)
fcntl (default: 1.0.0)
fiber-local (1.0.0)
fiddle (default: 1.0.0)
fileutils (default: 1.1.0)
fluent-plugin-opensearch (1.0.8)
fluent-plugin-slack (0.6.7)
fluentd (1.14.0)
forwardable (default: 1.2.0)
gdbm (default: 2.0.0)
http_parser.rb (0.7.0)
io-console (default: 0.4.7)
ipaddr (default: 1.2.2)
irb (default: 1.0.0)
jmespath (1.6.1)
json (2.4.1, default: 2.1.0)
logger (default: 1.3.0)
matrix (default: 0.1.0)
minitest (5.11.3)
msgpack (1.5.1)
multi_json (1.15.0)
multipart-post (2.2.3)
mutex_m (default: 0.1.0)
net-telnet (0.2.0)
nio4r (2.5.8)
oj (3.10.18)
opensearch-api (2.0.2)
opensearch-ruby (2.0.3)
opensearch-transport (2.0.1)
openssl (default: 2.1.2)
ostruct (default: 0.1.0)
power_assert (1.1.3)
prime (default: 0.1.0)
protocol-hpack (1.4.2)
protocol-http (0.21.0)
protocol-http1 (0.13.2)
protocol-http2 (0.14.2)
psych (default: 3.1.0)
rake (12.3.3)
rdoc (default: 6.1.2.1)
rexml (default: 3.1.9.1)
rss (default: 0.2.7)
ruby2_keywords (0.0.5)
scanf (default: 1.0.0)
sdbm (default: 1.0.0)
serverengine (2.2.5)
shell (default: 0.7)
sigdump (0.2.4)
stringio (default: 0.0.2)
strptime (0.2.5)
strscan (default: 1.0.0)
sync (default: 0.5.0)
test-unit (3.2.9)
thwait (default: 0.1.0)
timers (4.3.3)
tracer (default: 0.1.0)
tzinfo (2.0.4)
tzinfo-data (1.2022.1)
webrick (default: 1.4.4)
xmlrpc (0.3.0)
yajl-ruby (1.4.2)
zlib (default: 1.0.0)
```
* OpenSearch version (optional): 2.2.1
* OpenSearch template(s) (optional)
Contributor guide
Assessment
This issue has not been assessed yet.