Remove partial support for duplicate MultIindex names unless they are all None
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
Currently our MultiIndex class supports duplicate names, while DataFrames do not. The MultiIndex support is buggy, however, and we are frequently finding new edge cases that break it. Since pandas DataFrames do support duplicate names and [we explicitly choose not to](https://github.com/rapidsai/cudf/blob/branch-22.04/python/cudf/cudf/core/dataframe.py#L4377), I think it makes sense to do the same for MultiIndex. It improves our internal consistency and helps us write much more robust code. Making this change would probably fix a number of currently unknown/hidden bugs.
The major caveat here is that we do need to support MultiIndexes where all the names are None. However, handling this case would potentially be much simpler since we could use a sentinel or another class attribute to track whether names are meaningful or not. Default names could be integers, and any setting of names would require setting all column names to unique values.
An aside: If we ever did want to support duplicate names properly, it would involve a refactoring at the level of `ColumnAccessor`, which currently uses a dictionary as the underlying data structure to map names to columns. We would then need to update all of our functions that rely on `_from_data` to populate a new object that could support duplicate names rather than a dictionary. This is a substantial undertaking and out of scope for this issue.
Contributor guide
Assessment
This issue has not been assessed yet.