dotnet / dotnet/machinelearning
Add way to customize when cross validation is used / easier way to use cross validation with AutoML
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
### System information
AutoML 0.17.4
### Issue
After looking through the code it appears that cross validation will be used in AutoML IF you have less than 15,000 rows and you don't provide a custom validation set. I would make a couple of recommendations:
1) Document the 15,000 and the 10 fold somewhere beyond just in the source code. It was more difficult than it should be to find those 2 numbers
2) Allow the 15,000 and 10 fold to be customized. I realize that the 10 fold can be changed with an override of the execute method. However it then returns a CrossValidationExperimentResult which requires extensive custom code to retrieve scores averaged over the folds vs. when AutoML automatically chooses to utilize cross validation and generates summary ExperimentResult
Contributor guide
Assessment
This issue has not been assessed yet.