confluentinc / confluentinc/confluent-kafka-python
[Version 2.2.0] list_topics() messes up offsets
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
I know, we're using an old version, but can't upgrade due to dropped CentOS support.
Really weird behaviour....
What I do is roughly:
```python
consumer = make_consumer(enable_auto_commit=False)
consumer.subscribe(my_topic)
while True:
msgs = consumer.consume(timeout=timeout_sec, num_messages=max_records)
consumer.list_topics() # This is the crucial line - if present, next consume will be from committed offset, otherwise, will continue from last polled
consumer.commit(message=msgs[0])
```
Somehow `list_topics()` ruins the expected behaviour of polling the next batch every time - in the example it polls overlapping batches.
Interesting note: if I do `assign()` instead of `subscribe()` it works as expected even with `list_topics()` calls.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.