confluentinc / confluentinc/confluent-kafka-python
Producer goes to retry-backoff by send_offsets_to_transaction() thus degrading performance
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
Description
===========
For some reason the `send_offsets_to_transaction()` call will be 50+ times slower when there is more than one message in the input topic of the transactor.
How to reproduce
================
I've made a [reproduction demo](https://github.com/Traktormaster/kafka-tr-issue-201011) that has two modes of operation. In one the issue is present and in the other it is absent.
The [readme](https://github.com/Traktormaster/kafka-tr-issue-201011/blob/main/README.md) in that repository contains detailed information about the environment, the issue and the demo. There are log excerpts from the clients and the broker.
Checklist
=========
Please provide the following information:
- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): ('1.5.0', 17104896) ('1.5.0', 17105151)
- [x] Apache Kafka broker version: 2.5.0 or 2.6.0
- [x] Client configuration: *see reproduction demo*
- [x] Operating system: Linux 5.8.14
- [x] Provide client logs (with `'debug': '..'` as necessary): *see reproduction demo*
- [x] Provide broker log excerpts: *see reproduction demo*
- [x] Critical issue: nearly unusable performance when the issue is present
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.