aio-libs / aio-libs/aiokafka

A Consumer with no partitions assigned will timeout and leave the consumer group

Offen
#720 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
1.4k
Forks
269
Ø Merge
1 T. 1 Std.
Gemergte PRs (30 T.)
6

Beschreibung

**Describe the bug**

If a Consumer has no partitions assigned, it will eventually timeout and leave the consumer group, which means that it will not be available for assignment on future rebalances, and must be restarted to rejoin the group.

This code:

https://github.com/aio-libs/aiokafka/blob/5d9af4d3dee17f9a9cffdbd904823cf61e54b442/aiokafka/consumer/group_coordinator.py#L750

triggers the consumer to leave the group when the fetcher is idle for max_poll_seconds, but when there are no partitions assigned the fetcher will always be idle.

**Expected behaviour**

A Consumer with no partitions assigned should remain in the Consumer group and be available for assignment on future rebalances.

**Environment (please complete the following information):**
- aiokafka version: 0.7.0
- kafka-python version: 3.8
- Kafka Broker version (`kafka-topics.sh --version`): 4.0.0-5.3.1
- Other information (Confluent Cloud version, etc.):

**Reproducible example**
Bring up two consumers on a topic that has only one partition.

Run the command:

kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group --members

to see that 2 members are listed in the group but one has no partitions assigned.

After roughly 10 minutes the consumer that was not assigned a partition will leave the group and will no longer be available for assignment if the first consumer should fail.

Run the same command again and there is only one member shown in the group, assigned to the one partition.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.