confluentinc / confluentinc/confluent-kafka-python
Support for connect_cb
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
We have an experimental patch which adds support for setting `connect_cb` as a configuration, however looking back we found [this issue from 2017
](https://github.com/confluentinc/confluent-kafka-python/issues/219#issuecomment-315983155) which says that adding it would be hard.
And it is! By doing so we discovered a deadlock that would occur since the `connect_cb` is called in a different thread than the other callback configs due to in part the GIL being held during calls to `rd_kafka_assign`. Trying to solve this, we released the GIL for the duration of `rd_kafka_assign` and things seem to be working well now.
That context is here because I'm curious to know:
1. Is supporting `connect_cb` hard for other specific reasons?
2. Does the assign call need to hold the GIL for other safety reasons?
Basically I'm wondering if there are some invariants I may be stepping on with these changes. Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.