dotnet / dotnet/machinelearning

Add way to customize when cross validation is used / easier way to use cross validation with AutoML

Open
#5,626 0 comments 0 reactions 0 assignees View on GitHub
area-AutoML documentation Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

### System information
AutoML 0.17.4

### Issue

After looking through the code it appears that cross validation will be used in AutoML IF you have less than 15,000 rows and you don't provide a custom validation set. I would make a couple of recommendations:

1) Document the 15,000 and the 10 fold somewhere beyond just in the source code. It was more difficult than it should be to find those 2 numbers
2) Allow the 15,000 and 10 fold to be customized. I realize that the 10 fold can be changed with an override of the execute method. However it then returns a CrossValidationExperimentResult which requires extensive custom code to retrieve scores averaged over the folds vs. when AutoML automatically chooses to utilize cross validation and generates summary ExperimentResult

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.