fluent / fluent/fluent-plugin-kafka
Enormous amount of TIME_WAIT connections with non-buffered (direct) output
- Dominant language
- Ruby
- Stars
- 307
- Forks
- 178
- Avg merge
- 10h 37m
- Merged PRs (30d)
- 13
Description
I am using this configuration:
```
@type kafka
brokers somebrokers:9092
default_topic 'sometopic'
output_data_type json
max_send_retries 300
required_acks 0
```
And with this I do have a lot of TIME_WAIT connections to kafka node:
```
# netstat -tulnap | grep TIME_WAIT | wc -l
19699
```
```
# netstat -tulnap | grep TIME_WAIT
...
tcp 0 0 192.168.100.22:41907 192.168.100.55:9092 TIME_WAIT -
tcp 0 0 192.168.100.22:43715 192.168.100.55:9092 TIME_WAIT -
tcp 0 0 192.168.100.22:35498 192.168.100.55:9092 TIME_WAIT -
tcp 0 0 192.168.100.22:56291 192.168.100.55:9092 TIME_WAIT -
tcp 0 0 192.168.100.22:53716 192.168.100.55:9092 TIME_WAIT -
```
What could be the cause of fluentd keeping those connections alive until timeout? As fas as I've managed to debug fluentd is not reusing any of established connection in this mode as well.... With this is produces unreasonable load on network devices.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.