confluentinc / confluentinc/confluent-kafka-python

list_topics() crash

Open
#471 4 comments 0 reactions 0 assignees View on GitHub
bug investigate further
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========
Created a consumer, called list_topics(), immediate crash.

How to reproduce
================
There's a simple bug in the code, but it's only triggered by a somewhat bad installation environment. Let me explain: in my setup, I can import confluent_kafka, but if I do this:

>>> import confluent_kafka.admin
Traceback (most recent call last):
File "", line 1, in
File "/usr/anim/modsquad/oses/rhel7/src/confluent-kafka-python-v1.0.0-PRE1/build/lib.linux-x86_64-2.7/confluent_kafka/admin/__init__.py", line 20, in
import concurrent.futures
ImportError: No module named concurrent.futures
>>> import concurrent
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named concurrent

OK, that's on me (though I don't see anywhere that I should have the concurrent module). But given that I can't import confluent_kafka.admin (though I can import confluent_kafka), if you look in the code for list_topics(), you'll see that eventually this call is made:

cfl_PyObject_lookup("confluent_kafka.admin",
"ClusterMetadata");

and in the code for cfl_PyObject_lookup you call
PyObject *module = PyImport_ImportModule(modulename);

but don't ever check that module is non-null. so basically, a bad setup leads to an inscrutable crash. Not high priority, but thought I'd point it out. maybe raise an exception if the PyImport_ImportModule fails and tell me the module couldn't be loaded? would have saved an hours worth of digging.

That way, you don't have to tell me to install the concurrent module: the stack trace will provide me with a clue.

FYI, I only figured this out from looking at the code -- i didn't know to try importing the admin module till i saw it in the code and backtracked it all.

deb@pixar.com
David Baraff

Overall, love this code. thanks for it!

Checklist
=========
Please provide the following information:

- [ ] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):
- [ ] Apache Kafka broker version:
- [ ] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.