dotnet / dotnet/machinelearning
Upsampling with IDataView
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
Upsampling is a common practice for unbalanced data sets. The best practice for upsampling is to upsample the training set AFTER splitting into train and test sets, because if you duplicate rows before splitting, you'll get identical rows in the train and test set. This is data leakage from the train to the test set, and if you overfit the training set it will be partially hidden by the identical rows in the test set pumping up the scores.
There is no Upsampling transformer, so far as I can find. If I use the built-in TrainTestSplit method, I get two IDataViews back. Now I can't upsample by adding/duplicating rows in the training set because IDataView is immutable.
So basically I have to load from text file myself, because if I use TextLoader I get an IDataView which puts me in the same predicament. Next, I have to reproduce the functionality in TrainTestSplit(), in order to split my original dataset myself into train and test split. Next, I have to upsample the training set myself, and then remember to separately run both the train and test split through the regular data pipeline I've created.
I can't see how the CustomTransform can be used to Upsample. How would you suggest is the correct way to upsample a data set with ML.Net? Like a nice, convenient way. Not an arduous solution like I mentioned above, which will just drive people back to python instead.
Contributor guide
Assessment
This issue has not been assessed yet.