fluent / fluent/fluent-plugin-kafka

events distribution in kafka partitions

Open
#271 4 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Ruby
Stars
307
Forks
178
Avg merge
10h 37m
Merged PRs (30d)
13

Description

Hello

We are having unequal events distribution of events in the partitions. our topic have 8 partitions we are noticing that one of the partition getting bulk of the events others paritions are getting very less. please advise.

Below is our fluentd-kafka-output config
```

@type kafka_buffered
brokers broker1:9092,broker2:9092,broker3:9092
default_topic MyTopic
output_data_type msgpack
output_include_tag true
get_kafka_client_log false
buffer_type file
buffer_path /tmp/td-agent/buffer
flush_interval 3s
required_acks 1
num_threads 8

```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.