fluent / fluent/fluent-plugin-mongo

failed to flush the buffer. Mongo::Error::NoServerAvailable "No server is available matching preference: #<Mongo::ServerSelector::Primary:0x47353936741240"

Open
#147 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
175
Forks
56
PR merge metrics
No merged PRs in 30d

Description

Hey guys,

I'm trying to setup a pretty simple fluentd config. I simply want some logs to be sent to my replica set (which only consists of 1 primary which is a mongo:4.0.18 docker img) on my local kubernetes. The mongodb server is available on "mongodb.default:27017".

This is my fluentd config
```

@type http
port 8888
bind 0.0.0.0

@type json
time_type unixtime

#
# @type stdout
#

@type mongo_replset

nodes mongodb.default:27017,
replica_set rs0
database fluentd
collection ${tag}


time_key time


flush_interval 10s

log_level debug

```
I'm using this Dockerfile for my fluentd container:
```
FROM fluent/fluentd:latest

COPY setup.sh /tmp/
RUN /tmp/setup.sh

COPY fluentd.conf /fluentd/etc/
```
with `setup.sh` beeing:
```
apk add --no-cache --update --virtual .build-deps \
sudo build-base ruby-dev

# install plugins
sudo gem install fluent-plugin-mongo

# clean up
sudo gem sources --clear-all
apk del .build-deps
rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
```

Apparently this doesn't work though. I'm always getting this error:
```
2020-10-21 13:20:17 +0000 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2020-10-21 13:20:18 +0000 chunk="5b22e330161c80e2e79154b8aea6ff51" error_class=Mongo::Error::NoServerAvailable error="No server is available matching preference: # using server_selection_timeout=30 and local_threshold=0.015"
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/mongo-2.6.4/lib/mongo/server_selector/selectable.rb:115:in `select_server'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/mongo-2.6.4/lib/mongo/cluster.rb:263:in `next_primary'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo.rb:255:in `list_collections_enabled?'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo.rb:259:in `collection_exists?'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo.rb:273:in `get_collection'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo.rb:305:in `operate'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo_replset.rb:40:in `block in operate'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo_replset.rb:47:in `rescue_connection_failure'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo_replset.rb:39:in `operate'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo.rb:186:in `write'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-mongo-1.4.1/lib/fluent/plugin/out_mongo_replset.rb:33:in `write'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.3.2/lib/fluent/plugin/output.rb:1123:in `try_flush'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.3.2/lib/fluent/plugin/output.rb:1423:in `flush_thread_run'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.3.2/lib/fluent/plugin/output.rb:452:in `block (2 levels) in start'
2020-10-21 13:20:17 +0000 [warn]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.3.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
```

Things I already tried:
1. I tried to use `@type mongo` instead. used `connection_string` instead of the standalone params but everything leads to the same error.
2. I downgraded my mongodb from v4.2 initially to 4.0.18 now. Exactly the same error.

This is my first time working with fluentd and the mongo-plugin. Also I don't have any experience with ruby. Any ideas / suggestions are appreciated!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.