confluentinc / confluentinc/confluent-kafka-python
Producer silently swallows produce error if it does not have Write permissions
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
Description
===========
We have a producer that produces to a topic and does not have Write permissions on it. If I would produce to it, I would expect the produce call to throw an exception. Instead, librdkafka only logs `PARTCNT|rdkafka#producer-3| [thrd:ssl://localhost:9090/]: Topic test-topic partition count changed from 3 to 0`.
I do produce an `on_delivery` callback and also call `producer.flush()` and `producer.poll()` after the produce.
I found some older related issues https://github.com/confluentinc/confluent-kafka-python/issues/738 and https://github.com/confluentinc/confluent-kafka-python/issues/704 which are already closed with `v1.6.0, will provide improved authentication error reporting`. There is also this recently opened issue https://github.com/confluentinc/confluent-kafka-python/issues/1787 for a similar problem on the consumer.
How to reproduce
================
Initialize a producer without Write permissions and then run
```
def delivery_report(err: KafkaError, msg: Message) -> None:
raise RuntimeError(err)
producer.produce("topic", "value", on_delivery=delivery_report)
producer.flush(timeout=1.0)
producer.poll(timeout=0)
```
You will observe the log message, but no error will be raised.
Checklist
=========
Please provide the following information:
- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): 2.6.0
- [x] Apache Kafka broker version: 3.7.1
- [x] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [x] Critical issue
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.