[RFC] Deprecate the `Slice` method of `DMatrix`.
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
This is a proposal that we delegate data slicing to external libraries.
## Motivation
`DMatrix` is an internal data interface, used for all XGBoost algorithm. The `DMatrix` object has a few different representation, based on which some more variants can be generated at runtime. Currently we have 3 different base representation for `DMatrix`, namely a CSR like `SparsePage`, a Ellpack sparse matrix for storing GPU histogram index, and lastly external memory batched support for above two in-memory formats.
The `Slice` method for `DMatrix` is mostly used for early stopping where we need to split data for folds. But the implementation is rather insufficient for many use cases as it supports only the `SparsePage` and without shuffling support, not to mention distributed environment support. Also when used with ranking, it doesn't maintain the `group_ptr_`, which is delegated to a higher level Python wrapper script, resulting in an inconsistent support status for different language bindings.
## Moving forward
It has became clear that the cross validation implementation in xgboost is here to stay due to callback feature. So a better way to reduce complexity in `DMatrix` is to seek possible implementation of cv without having to slice up `DMatrix`. It might be possible that we can slice up input data instead.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.