how to handle UnknownTopicOrPartitionError?
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 269
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 6
Description
What I want to do is subscribing topics that have name with specific pattern like `email.service.*` dynamically.
that dynamically means that while it is consuming I should be able to create or delete topics and it should handle it.
Creating a topic and subscribing it dynamically was fine..
However if I delete a topic that already being subscribed, consumer spew out a lot of error messages like follows:
```
OffsetCommit failed for group email.service on partition TopicPartition(topic='email.service.order', partition=0) with offset OffsetAndMetadata(offset=1, metadata=''): UnknownTopicOrPartitionError
OffsetCommit failed for group email.service on partition TopicPartition(topic='email.service.order', partition=0) with offset OffsetAndMetadata(offset=1, metadata=''): UnknownTopicOrPartitionError
OffsetCommit failed for group email.service on partition TopicPartition(topic='email.service.order', partition=0) with offset OffsetAndMetadata(offset=1, metadata=''): UnknownTopicOrPartitionError
...
```
But I couldn't handle this, because it didn't raise any exceptions. So I checked the code.
https://github.com/aio-libs/aiokafka/blob/724e8105ce0163b2b2df176af65605aafd3ed060/aiokafka/consumer/group_coordinator.py#L1146-L1149
for UnknownTopicOrPartitionError, it just warn it not raise it. How can I handle this exception?
if I handled this exception, I would go like this:
```python
...
try:
result = await consumer.getmany(timeout_ms=500)
except UnknownTopicOrPartitionError as e:
consumer.unsubscribe()
consumer.subscribe(pattern='email.service.*')
...
```
maybe this is a xy problem, if are there any better way to subscribe topics dynamically, please let me know!
thanks!
Contributor guide
Assessment
This issue has not been assessed yet.