NVIDIA / NVIDIA/cudf

[FEA] Support duplicate column labels in cudf.DataFrame

Open
#16,533 1 comment 0 reactions 0 assignees View on GitHub
cudf.pandas feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
pandas currently supports duplicate column labels when constructing a `DataFrame` e.g. `pandas.DataFrame([[0, 1]], columns=[0, 0])` and operations with these duplicate labels (indexing, index to column level transferring, etc)

cudf currently does not support this because all public data structures are essentially represented as a mapping of `label -> column` which requires that column labels are unique. Therefore, there are many places where we must check for duplicate labels and raise a `ValueError`/`NotImplementedError` (for eventual fallback in `cudf.pandas`)

(This also applies to a `MultiIndex` which has "partial" support for duplicate names xref https://github.com/rapidsai/cudf/issues/10500)

**Describe the solution you'd like**
It would be great to support duplicate column labels without having to fall back to `cudf.pandas`. The most "minimal" change would probably have the `ColumnAccessor` also carry of `dict[Hashable, list[int]]` of `column label -> integer positions` and have the mapping of columns be `dict[int, ColumnBase]` of `integer position -> Column`

**Describe alternatives you've considered**
Status quo and fall back to `cudf.pandas` for this case.

**Additional context**
xref https://github.com/rapidsai/cudf/pull/16514

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.