[FEA] Support duplicate column labels in cudf.DataFrame
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
pandas currently supports duplicate column labels when constructing a `DataFrame` e.g. `pandas.DataFrame([[0, 1]], columns=[0, 0])` and operations with these duplicate labels (indexing, index to column level transferring, etc)
cudf currently does not support this because all public data structures are essentially represented as a mapping of `label -> column` which requires that column labels are unique. Therefore, there are many places where we must check for duplicate labels and raise a `ValueError`/`NotImplementedError` (for eventual fallback in `cudf.pandas`)
(This also applies to a `MultiIndex` which has "partial" support for duplicate names xref https://github.com/rapidsai/cudf/issues/10500)
**Describe the solution you'd like**
It would be great to support duplicate column labels without having to fall back to `cudf.pandas`. The most "minimal" change would probably have the `ColumnAccessor` also carry of `dict[Hashable, list[int]]` of `column label -> integer positions` and have the mapping of columns be `dict[int, ColumnBase]` of `integer position -> Column`
**Describe alternatives you've considered**
Status quo and fall back to `cudf.pandas` for this case.
**Additional context**
xref https://github.com/rapidsai/cudf/pull/16514
Contributor guide
Assessment
This issue has not been assessed yet.