dotnet / dotnet/machinelearning

Upsampling with IDataView

Open
#4,028 5 comments 0 reactions 0 assignees View on GitHub
area-Core enhancement Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Upsampling is a common practice for unbalanced data sets. The best practice for upsampling is to upsample the training set AFTER splitting into train and test sets, because if you duplicate rows before splitting, you'll get identical rows in the train and test set. This is data leakage from the train to the test set, and if you overfit the training set it will be partially hidden by the identical rows in the test set pumping up the scores.

There is no Upsampling transformer, so far as I can find. If I use the built-in TrainTestSplit method, I get two IDataViews back. Now I can't upsample by adding/duplicating rows in the training set because IDataView is immutable.

So basically I have to load from text file myself, because if I use TextLoader I get an IDataView which puts me in the same predicament. Next, I have to reproduce the functionality in TrainTestSplit(), in order to split my original dataset myself into train and test split. Next, I have to upsample the training set myself, and then remember to separately run both the train and test split through the regular data pipeline I've created.

I can't see how the CustomTransform can be used to Upsample. How would you suggest is the correct way to upsample a data set with ML.Net? Like a nice, convenient way. Not an arduous solution like I mentioned above, which will just drive people back to python instead.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.