automl / automl/AutoFolio

Make number of CV folds configurable

Open
#15 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
43
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Hi Marius,

when using the CSV-facade, one can specify the CV-folds with an extra CSV file:
https://github.com/automl/AutoFolio/blob/f296f528b1b684d36837075b0e8160e3fa4124f7/autofolio/facade/af_csv_facade.py#L27
However, when not doing that, it will call the ASLibScenario without specifying the number of CV folds:
https://github.com/mlindauer/ASlibScenario/blob/9ce38dd43dc6e2834cb8ef44cd837514e950a9c5/aslib_scenario/aslib_scenario.py#L168
In that case, the default of 10 folds will be used:
https://github.com/mlindauer/ASlibScenario/blob/9ce38dd43dc6e2834cb8ef44cd837514e950a9c5/aslib_scenario/aslib_scenario.py#L827

It would be convenient, if one could, instead of having to create a full CVS specifying the splits, just pass a custom number of CV folds to use AutoFolio more easily with different dataset sizes.

Let me know if you'd like me to provide a pull request.

Best,
Jan

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in autofolio/facade/af_csv_facade.py at the CSV-facade handling, then inspect the ASLibScenario call and its default fold behavior at the linked locations. Determine where AutoFolio accepts configuration and verify that a custom fold count can be used without supplying a full CSV; done means different dataset sizes can select their desired number of CV folds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.