dotnet / dotnet/machinelearning

TreeFeaturizer w/ Subsampling

Open
#3,808 0 comments 0 reactions 1 assignee Claimed by @wschin View on GitHub
area-Trees enhancement Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

To support TB+ scale datasets, we should have a row sub-sampling parameter in the Tree Featurizer.

Why `TreeFeat`? The Tree Featurizer learns the feature interactions on a sub-set of the dataset. The `leaves` output of Tree Featurizer is then sent to a linear model (SDCA/AP/etc) which is scalable to TB scale datasets.

**Gain**:
This provides an end-to-end solution for scalable learning. You get the accuracy of the Tree model plus the streamability of the linear model.

**Request**:
* Provide a `subsample` parameter for `TreeFeat`
* Inside `TreeFeat` run the `TrainTestSplit` to get the correct subsample percent

**Other solutions**:
* TrainTest split to create a sub-sample of the dataset before `TreeFeat`, and train `TreeFeat` on this subsample. This fails in CV since you need a singular dataview to train on.
* Shuffle + Take. This fails since our shuffle is block-wise; the 1st 4k rows are shuffled against each other, then the next 4k rows are independently shuffled against each other. This leads to the Take only receiving rows from the top of the file. Also fails in CV.
* Generate + RangeFilter. We hid our Generate transform. Also fails in CV.

We would like to use the `TreeFeat` plus a streamable Linear model in AutoML to provide a good model to the user on datasets larger than RAM.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.