Hyperband: training for a longer time
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
Hyperband's rule of thumb works great when I want to sample more parameters than training epochs:
``` python
>>> X, y = make_classification(n_samples=1000)
>>> n_params = 200
>>> n_examples = len(X) * 200
>>> max_iter = n_params
>>> chunksize = n_examples // n_params
>>> chunksize
1000
```
This indicates the model should see the entire dataset every `partial_fit` call:
``` python
>>> from dask_ml.model_selection import HyperbandSearchCV
>>> search = HyperbandSearchCV(model, params, max_iter=200)
>>> search.metadata["n_models"]
143
```
How do I train for longer while still sampling (about) 200 parameters?
---
The rule of thumb won't work because it specifies a chunk size larger than the array. Right now, the only way is to sample more parameters:
``` python
>>> from dask_ml.model_selection import HyperbandSearchCV
>>> search = HyperbandSearchCV(model, params, max_iter=400) # to train for no more than 400 epochs
>>> search.metadata["n_models"]
415
```
That's more hyperparameters than I need to sample.
Contributor guide
Research direction
Start with the HyperbandSearchCV entry point and reproduce the issue's max_iter=200 and max_iter=400 examples. Trace how max_iter affects training duration and metadata["n_models"]; done means allowing longer training while sampling about 200 parameters, with coverage for the requested behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100