confluentinc / confluentinc/confluent-kafka-python

Producer silently swallows produce error if it does not have Write permissions

Open
#1,847 3 comments 1 reaction 0 assignees View on GitHub
enhancement priority:high status:duplicate
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========
We have a producer that produces to a topic and does not have Write permissions on it. If I would produce to it, I would expect the produce call to throw an exception. Instead, librdkafka only logs `PARTCNT|rdkafka#producer-3| [thrd:ssl://localhost:9090/]: Topic test-topic partition count changed from 3 to 0`.

I do produce an `on_delivery` callback and also call `producer.flush()` and `producer.poll()` after the produce.

I found some older related issues https://github.com/confluentinc/confluent-kafka-python/issues/738 and https://github.com/confluentinc/confluent-kafka-python/issues/704 which are already closed with `v1.6.0, will provide improved authentication error reporting`. There is also this recently opened issue https://github.com/confluentinc/confluent-kafka-python/issues/1787 for a similar problem on the consumer.

How to reproduce
================
Initialize a producer without Write permissions and then run
```
def delivery_report(err: KafkaError, msg: Message) -> None:
raise RuntimeError(err)

producer.produce("topic", "value", on_delivery=delivery_report)
producer.flush(timeout=1.0)
producer.poll(timeout=0)
```

You will observe the log message, but no error will be raised.

Checklist
=========
Please provide the following information:

- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): 2.6.0
- [x] Apache Kafka broker version: 3.7.1
- [x] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [x] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.