confluentinc / confluentinc/confluent-kafka-python
Producer hide librdkafka failures and silently return None as message on poll
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
Description
===========
In case an error happened on Kerberos request authentification `librdkafka` will provide some logs which also are not returned into python space and for just `None` returned from message poll without an error indication.
How to reproduce
================
```python
consumer_config = {
'bootstrap.servers': 'kafka-int.epm-eco.projects.example.com:9095}',
'schema.registry.url': 'http://schema-registry-int.epm-eco.projects.example.com:8081',
'group.id': 'epm-ddo.consumer.2',
'auto.offset.reset': 'earliest',
'security.protocol': 'SASL_SSL',
'ssl.ca.location': 'conf/integration/epm-eco-int.ca.crt',
'sasl.mechanisms': 'GSSAPI',
'sasl.kerberos.principal': 'kafkaclient',
'sasl.kerberos.kinit.cmd': 'bash -c "cat conf/integration/paswd | /usr/bin/kinit Pavel_Alexeev@PETERSBURG.EXAMPLE.COM"',
'sasl.kerberos.service.name': 'kafka',
# 'log_level': 7, 'debug': 'all' ## Logs will be ONLY in container output
}
topic = '....cdm.LocationChartNode'
consumer = AvroConsumer(consumer_config)
consumer.subscribe([topic])
message = consumer.poll(5)
print(message)
consumer.close()
```
Printed result `None`, if look into the container, `librdkafka` produce errors like:
```
%3|1628784442.369|SASLREFRESH|rdkafka#consumer-34| [thrd:main]: Kerberos ticket refresh failed: bash -c "cat conf/integration/paswd | /usr/bin/kinit Pavel_Alexeev@PETERSBURG.EXAMPLE.COM": exited with code 1
```
Checklist
=========
Please provide the following information:
- [✔] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):
('1.7.0', 17235968), ('1.7.0', 17236223)
- [✔] Apache Kafka broker version: 5.3.2 (uncertain)
- [✔] Client configuration: Kerberros (SASL_SSL), see full config above in reproduce section
- [✔] Operating system: Linux. Fedora and Ubuntu checked
- [✔] Provide client logs (with `'debug': '..'` as necessary)
- [✔] Provide broker log excerpts: sems irrelevant andI have no access
- [✔] Critical issue: yes. Silently skip errors behavior do not allow rely on behaviour and understand what is there something go wrong or just end of topic reached
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.