confluentinc / confluentinc/confluent-kafka-python

Provide proper documentation and type hints for supported config options

Open
#1,683 1 comment 0 reactions 0 assignees View on GitHub
docs enhancement priority:high status:waiting-for-interest
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========

The documentation for the Python library refers to librdkafka's [CONFIGURATION.md](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md) file when it comes to available config options.

However, this is very far from ideal, as it is not immediately obvious, which C types map to which Python types. Furthermore, this prevents autocompletion of configuration properties.

It would be nice if:

1. The full configuration is documented for the Python library as well, listing Python types which are required for each property.
2. The C config is mapped to a Python config TypedDict or Pydantic type, which is then made available through this library. Furthermore, using "." as a separator feels unnatural in Python, where snake case with underscores is preferred.
3. The config used more precise type annotations, which are then internally mapped to the C config. E.g. bootstrap.servers would possibly be a list[str] instead of a str.

How to reproduce
================

n.a.

Checklist
=========
Please provide the following information:

- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): ('2.3.0', 33751040), ('2.3.0', 33751295)
- [ ] Apache Kafka broker version: n.a.
- [ ] Client configuration: `{...}`: n.a.
- [x] Operating system: macOS 14.1.1 (23B81)
- [ ] Provide client logs (with `'debug': '..'` as necessary): n.a.
- [ ] Provide broker log excerpts: n.a.
- [ ] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.