NVIDIA / NVIDIA/cudf

[FEA] Consider removing tight pinning on librdkafka

Open
#21,957 0 comments 0 reactions 0 assignees View on GitHub
conda cuIO feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

It appears we've historically had a tight pinning on `librdkafka` used in `libcudf_kafka` (probably to ensure build stability of `libcudf_kafka`?). However as temporarily solved in https://github.com/rapidsai/cudf/pull/21951, this tight pinning is not guaranteed to be solvable in a conda environment overtime as we continue to support more Python versions.

**Describe the solution you'd like**
AFAICT we only use the `librdkafka`'s [public API](https://github.com/confluentinc/librdkafka?tab=readme-ov-file#documentation) in libcudf_kafka so if we trust the API stability (there were no API changes when upgrading in https://github.com/rapidsai/cudf/pull/21951), it should be safe to just unpin `librdkafka`

**Describe alternatives you've considered**
* Status quo
* Consider deprecating (lib)cudf_kafka all together?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.