fluent / fluent/fluent-plugin-multiprocess
We're using the fluent-plugin-multiprocess plugin and sometimes td-agent binds to 0.0.0.0 instead of 127.0.0.1
- Dominant language
- Ruby
- Stars
- 44
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
Hi-
We're using the fluent-plugin-multiprocess plugin and sometimes td-agent binds to 0.0.0.0 instead of 127.0.0.1, when we have the following specified in our configs:
> ```
> ad-events:
> # Defines the content of the source section
> source: {
> type: 'forward',
> port: 24221,
> bind: '127.0.0.1',
> tag: 'gg.adevent'
> }
> # Defines the content of the match section
> match:
> general: {
> type: forward,
> buffer_type: memory,
> buffer_queue_limit: 128,
> send_timeout: 10s,
> recover_wait: 10s,
> heartbeat_interval: 2s,
> phi_threshold: 5,
> hard_timeout: 10s,
> flush_interval: 5s,
> retry_limit: 2
> }
> subsections:
> server: [
> {
> name: "aggregator-{{ fluentd_aggregator_ips[0] }}",
> host: "{{ fluentd_aggregator_ips[0] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[1] }}",
> host: "{{ fluentd_aggregator_ips[1] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[2] }}",
> host: "{{ fluentd_aggregator_ips[2] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[3] }}",
> host: "{{ fluentd_aggregator_ips[3] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[4] }}",
> host: "{{ fluentd_aggregator_ips[4] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[5] }}",
> host: "{{ fluentd_aggregator_ips[5] }}",
> weight: 33,
> port: 24221
> }
> ]
> # Secondary section will forward back to a local backup socket
> # Which will trigger the standalone mode (Collect and S3 upload)
> # On the client
> secondary: [
> {
> type: 'forward',
> host: '127.0.0.1',
> port: 34221
> }
> ]
> ```
This prevents things from actually shipping logs for us on that specific port, in those times td-agent binds to 0.0.0.0 instead of localhost 127.0.0.1. The issue recurs often, but is not consistently reproducible for us and the logs just say:

> `2016-07-31 12:47:07 -0700 [error]: unexpected error error_class=Errno::EADDRINUSE error=#`
The "fix"/remediation for the issue has been to just stop/start td-agent, which then allows td-agent to bind to localhost instead of 0.0.0.0.
The versions from the startup of the agent are:
> 2016-07-31 12:46:52 -0700 [info]: reading config file path="/etc/td-agent/td-agent.conf"
> 2016-07-31 12:46:52 -0700 [info]: starting fluentd-0.12.26
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-mixin-config-placeholders' version '0.4.0'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-datacounter' version '0.4.5'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-mongo' version '0.7.13'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-multiprocess' version '0.2.1'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-s3' version '0.6.8'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-td' version '0.10.28'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.2'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-webhdfs' version '0.4.2'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluentd' version '0.12.26'
> 2016-07-31 12:46:53 -0700 [info]: adding source type="multiprocess"
> 2016-07-31 12:46:53 -0700 [info]: using configuration file:
If you have and advice on what the issue might be, it'd be much appreciated.
Best regards!!!!
Jonathan
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.