Make number of CV folds configurable
- Dominant language
- Python
- Stars
- 43
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
Hi Marius,
when using the CSV-facade, one can specify the CV-folds with an extra CSV file:
https://github.com/automl/AutoFolio/blob/f296f528b1b684d36837075b0e8160e3fa4124f7/autofolio/facade/af_csv_facade.py#L27
However, when not doing that, it will call the ASLibScenario without specifying the number of CV folds:
https://github.com/mlindauer/ASlibScenario/blob/9ce38dd43dc6e2834cb8ef44cd837514e950a9c5/aslib_scenario/aslib_scenario.py#L168
In that case, the default of 10 folds will be used:
https://github.com/mlindauer/ASlibScenario/blob/9ce38dd43dc6e2834cb8ef44cd837514e950a9c5/aslib_scenario/aslib_scenario.py#L827
It would be convenient, if one could, instead of having to create a full CVS specifying the splits, just pass a custom number of CV folds to use AutoFolio more easily with different dataset sizes.
Let me know if you'd like me to provide a pull request.
Best,
Jan
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in autofolio/facade/af_csv_facade.py at the CSV-facade handling, then inspect the ASLibScenario call and its default fold behavior at the linked locations. Determine where AutoFolio accepts configuration and verify that a custom fold count can be used without supplying a full CSV; done means different dataset sizes can select their desired number of CV folds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100