aio-libs / aio-libs/aiokafka

UnknownTopicOrPartitionError cause infinite loop inside aiokafka.consumer.group_coordinator GroupCoordinator class, commit_offsets function

Open
#1,095 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
269
Avg merge
1d 1h
Merged PRs (30d)
6

Description

**Describe the bug**
I am using async consumer with subscribe by pattern.
I need to support working against topics which may added and removed over time.
When we remove a topic, an UnknownTopicOrPartitionError catch inside the commit_offsets function,
since it is retriable, the function sleeps and then retries again (forever)

**Expected behaviour**
Between each retry (exponential back-off) the system will force metadata update on the consumer.
After few retries, the system will raise a different error (which is not retriable) to up in the chain until it is gets to the source caller.

**Environment (please complete the following information):**
- aiokafka version: 0.12.0
- Kafka Broker version: NA
- Other information: NA

**Reproducible example**
```python
# Add a short Python script or Docker configuration that can reproduce the issue.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.