dotnet / dotnet/machinelearning
TreeFeaturizer w/ Subsampling
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
To support TB+ scale datasets, we should have a row sub-sampling parameter in the Tree Featurizer.
Why `TreeFeat`? The Tree Featurizer learns the feature interactions on a sub-set of the dataset. The `leaves` output of Tree Featurizer is then sent to a linear model (SDCA/AP/etc) which is scalable to TB scale datasets.
**Gain**:
This provides an end-to-end solution for scalable learning. You get the accuracy of the Tree model plus the streamability of the linear model.
**Request**:
* Provide a `subsample` parameter for `TreeFeat`
* Inside `TreeFeat` run the `TrainTestSplit` to get the correct subsample percent
**Other solutions**:
* TrainTest split to create a sub-sample of the dataset before `TreeFeat`, and train `TreeFeat` on this subsample. This fails in CV since you need a singular dataview to train on.
* Shuffle + Take. This fails since our shuffle is block-wise; the 1st 4k rows are shuffled against each other, then the next 4k rows are independently shuffled against each other. This leads to the Take only receiving rows from the top of the file. Also fails in CV.
* Generate + RangeFilter. We hid our Generate transform. Also fails in CV.
We would like to use the `TreeFeat` plus a streamable Linear model in AutoML to provide a good model to the user on datasets larger than RAM.
Contributor guide
Assessment
This issue has not been assessed yet.