NVIDIA / NVIDIA/cudf

[ENH] Audit cudf APIs for use of inappropriate algorithms

Open
#14,479 0 comments 3 reactions 0 assignees View on GitHub
feature request no-oom Performance Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

Historically (I think) certain features were available in libcudf before others. For example, while hash joins appeared quite early on `cudf::contains` was only factored out of the semi join infrastructure in #11100.

As a result, there are a number of places in cudf where an API was implemented using a sub-optimal approach (be that in terms of memory footprint or performance) just because it was needed in the Python API.

For example in #14478, we replace a sub-optimal (in both memory _and_ performance) inner join, with a call to `cudf::contains` now that it is available.

**Describe the solution you'd like**

We should go through and check for other instances of this historical anti-pattern and either:

- replace with calls to appropriate (existing) libcudf primitives
- gather feature requests for new libcudf primitives based on the usage we observe.

**Describe alternatives you've considered**

n/a

**Additional context**

It is probable that candidates can be found by looking calls to `merge` in the cudf codebase. As well as argsort/scatter/gather patterns (that's #13557).
- [ ] https://github.com/rapidsai/cudf/issues/13557
- [ ] https://github.com/rapidsai/cudf/pull/14478
- [ ] https://github.com/rapidsai/cudf/issues/14480
- [ ] https://github.com/rapidsai/cudf/issues/14485
- [ ] https://github.com/rapidsai/cudf/issues/14486
- [ ] https://github.com/rapidsai/cudf/issues/13630
- [ ] https://github.com/rapidsai/cudf/issues/13565
- [ ] https://github.com/rapidsai/cudf/issues/13456
- [ ] https://github.com/rapidsai/cudf/issues/14487

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.