fluent / fluent/fluentd

fluentd can't route tcp connection to workers evenly

Open
#2,172 5 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Check [CONTRIBUTING guideline](https://github.com/fluent/fluentd/blob/master/CONTRIBUTING.md) first and here is the list to help us investigate the problem.

- fluentd or td-agent version.
**fluentd 1.2.6**
- Environment information, e.g. OS.
Linux g9csf002-0-0-1 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
- Your configuration
```

@type tcp
port 8514
bind 0.0.0.0
format /(^(?[^\{]+)?(?\{.+type.+\})$)|(^(?[^\{].+))/
tag fluent.tst

@type rewrite_tag_filter

key message
pattern .+
tag logging.json

@type parser
key_name message

@type json
time_key time
time_format %iso8601
keep_time_key true

...

workers 10
log_level error

```
- Your problem explanation. If you have an error logs, write it together.
I started a fluentd cluster with 10 workers, and started 10 clients to connect to the fluentd clusters. I found just three of the workers took 10 connections from the clients. The traffic/tcp connections are not routed to the workers evenly.

```
top - 06:46:03 up 119 days, 34 min, 7 users, load average: 1.29, 0.61, 0.40
Tasks: 476 total, 1 running, 475 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.7 us, 0.1 sy, 0.0 ni, 95.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 13173099+total, 11717929+free, 8979432 used, 5572260 buff/cache
KiB Swap: 4194300 total, 4194300 free, 0 used. 12184173+avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6671 td-agent 20 0 532640 192444 5332 S 100.1 0.1 4:44.06 ruby
6669 td-agent 20 0 440480 161384 5320 S 60.9 0.1 3:28.13 ruby
6672 td-agent 20 0 428192 169652 5340 S 22.8 0.1 2:35.83 ruby
6670 td-agent 20 0 348320 125116 5284 S 0.2 0.1 0:05.38 ruby
6660 td-agent 20 0 264384 54132 1960 S 0.1 0.0 0:00.73 fluentd
6665 td-agent 20 0 348320 125084 5268 S 0.1 0.1 0:05.53 ruby
6666 td-agent 20 0 428192 169732 5348 S 0.1 0.1 3:04.13 ruby
6667 td-agent 20 0 348320 125076 5264 S 0.1 0.1 0:05.52 ruby
6668 td-agent 20 0 348320 125120 5284 S 0.1 0.1 0:05.43 ruby
6673 td-agent 20 0 348320 125112 5284 S 0.1 0.1 0:05.52 ruby
6674 td-agent 20 0 348320 125080 5264 S 0.1 0.1 0:05.68 ruby
```
Any ideas about this?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.