[Story]: Reworking of `iloc` and `loc` indexing
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
## Status quo
Indexing of dataframes and series happens through six user-facing routes:
- `DataFrame.__setitem__`/`DataFrame.__getitem__`
- `DataFrame.iloc.__setitem__`/`DataFrame.iloc.__getitem__`
- `DataFrame.loc.__setitem__`/`DataFrame.loc.__getitem__`
- `Series.__setitem__`/`Series.__getitem__`
- `Series.iloc.__setitem__`/`Series.iloc.__getitem__`
- `Series.loc.__setitem__`/`Series.loc.__getitem__`
These all have slightly different semantics (to match pandas behaviour), but there is still quite a lot of (possibly unnecessary) code duplication and a number of bugs around indexing. Many of these look to be because the business logic of handling slicing/gather-by-mask/indexing is intertwined with error handling and determining exactly what to slice. There's also logic effectively repeated between the loc and iloc versions in both cases.
It would be nice if the number of different paths into indexing was reduced, perhaps it is a pipe dream to share between Series and DataFrame (since a DataFrame is not just a collection of Series), but it feels like it should be possible to share more between iloc/loc/__setgetitem__.
Related issues:
### `iloc` bugs
- [ ] #12748
- [ ] #13013
- [ ] #13015
- [ ] #13265
- [ ] #13266
- [ ] #13267
- [ ] #13515
- [ ] #13293
### Index bugs
- [ ] #12954
### `loc` bugs
- [ ] #7448
- [ ] #8585
- [ ] #8693
- [ ] #11298
- [ ] #11944
- [ ] #12259
- [ ] #12286
- [ ] #12504
- [ ] #12505
- [ ] #12801
- [ ] #12833
- [ ] #13014
- [ ] #13015
- [ ] #13031
- [ ] #13268
- [ ] #13269
- [ ] #13270
- [ ] #13379
- [ ] https://github.com/rapidsai/cudf/issues/13653
- [ ] https://github.com/rapidsai/cudf/issues/13658
- [ ] https://github.com/rapidsai/cudf/issues/13652
### `at`/`iat` bugs
- [x] https://github.com/rapidsai/cudf/issues/20561
### Views vs. copies
- [ ] #7374
- [ ] #11085
- [x] #11990
### Other (mostly dtype-related)
- [ ] #2684
- [ ] #8184
- [ ] #11477
- [ ] #12039
- [ ] #13532
As we can see from this classification, `loc`-based indexing is definitely the harder nut to crack. The edge-cases that provoke most of the issues are cases where the values used in the indexing are _not_ in the index.
Contributor guide
Assessment
This issue has not been assessed yet.