NVIDIA / NVIDIA/cudf

[FEA] Either test or remove pandas compatibility mode

Open
#21,698 2 comments 0 reactions 0 assignees View on GitHub
cudf.pandas feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
pandas compatibility mode was originally introduced as a way to gate behaviors in cudf that were necessary for cudf.pandas but that we did not want to use in normal cudf. These behaviors generally fell into two categories: 1) incomplete features in cudf for which partial exposure in cudf.pandas was preferable, e.g. arrow dtype support; and 2) behavioral changes that had a negative performance impact that we did not want to impose on cudf users. As we are now moving towards greater pandas compatibility (see #18659), we are largely trying to move away from the first category: features should either be completely working in cudf, or they should fall back to pandas eagerly. Additionally, between cudf-polars and pylibcudf we have alternative engines for users who truly need absolute peak performance. Between these two factors, we may be able to remove pandas compatibility mode in the future by effectively having it always on. If we do want to limit the performance impacts to only cudf.pandas, however, we will need to test both code paths going forward.

**Describe the solution you'd like**
We should make a decision about the long-term future of pandas compatibility mode. If we want to keep it for performance reasons, we will need to include testing.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.