fluent / fluent/fluent-plugin-multiprocess

We're using the fluent-plugin-multiprocess plugin and sometimes td-agent binds to 0.0.0.0 instead of 127.0.0.1

Open
#10 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
44
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Hi-

We're using the fluent-plugin-multiprocess plugin and sometimes td-agent binds to 0.0.0.0 instead of 127.0.0.1, when we have the following specified in our configs:

> ```
> ad-events:
> # Defines the content of the source section
> source: {
> type: 'forward',
> port: 24221,
> bind: '127.0.0.1',
> tag: 'gg.adevent'
> }
> # Defines the content of the match section
> match:
> general: {
> type: forward,
> buffer_type: memory,
> buffer_queue_limit: 128,
> send_timeout: 10s,
> recover_wait: 10s,
> heartbeat_interval: 2s,
> phi_threshold: 5,
> hard_timeout: 10s,
> flush_interval: 5s,
> retry_limit: 2
> }
> subsections:
> server: [
> {
> name: "aggregator-{{ fluentd_aggregator_ips[0] }}",
> host: "{{ fluentd_aggregator_ips[0] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[1] }}",
> host: "{{ fluentd_aggregator_ips[1] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[2] }}",
> host: "{{ fluentd_aggregator_ips[2] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[3] }}",
> host: "{{ fluentd_aggregator_ips[3] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[4] }}",
> host: "{{ fluentd_aggregator_ips[4] }}",
> weight: 33,
> port: 24221
> },
> {
> name: "aggregator-{{ fluentd_aggregator_ips[5] }}",
> host: "{{ fluentd_aggregator_ips[5] }}",
> weight: 33,
> port: 24221
> }
> ]
> # Secondary section will forward back to a local backup socket
> # Which will trigger the standalone mode (Collect and S3 upload)
> # On the client
> secondary: [
> {
> type: 'forward',
> host: '127.0.0.1',
> port: 34221
> }
> ]
> ```

This prevents things from actually shipping logs for us on that specific port, in those times td-agent binds to 0.0.0.0 instead of localhost 127.0.0.1. The issue recurs often, but is not consistently reproducible for us and the logs just say:
![image](https://cloud.githubusercontent.com/assets/3851599/17313389/082c6728-5810-11e6-8597-c61bf7ab0f9c.png)

> `2016-07-31 12:47:07 -0700 [error]: unexpected error error_class=Errno::EADDRINUSE error=#`

The "fix"/remediation for the issue has been to just stop/start td-agent, which then allows td-agent to bind to localhost instead of 0.0.0.0.

The versions from the startup of the agent are:

> 2016-07-31 12:46:52 -0700 [info]: reading config file path="/etc/td-agent/td-agent.conf"
> 2016-07-31 12:46:52 -0700 [info]: starting fluentd-0.12.26
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-mixin-config-placeholders' version '0.4.0'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-datacounter' version '0.4.5'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-mongo' version '0.7.13'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-multiprocess' version '0.2.1'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-s3' version '0.6.8'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-td' version '0.10.28'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.2'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluent-plugin-webhdfs' version '0.4.2'
> 2016-07-31 12:46:53 -0700 [info]: gem 'fluentd' version '0.12.26'
> 2016-07-31 12:46:53 -0700 [info]: adding source type="multiprocess"
> 2016-07-31 12:46:53 -0700 [info]: using configuration file:

If you have and advice on what the issue might be, it'd be much appreciated.
Best regards!!!!
Jonathan

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.