Release finished model/job during cross validation
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
I'm wondering if there is a way to release the job/worker when a fitting job finishes during RandomizedSearchCV.
I'm doing something like
```
random_search_cv = RandomizedSearchCV(
estimator=model,
param_distributions=params_grid,
n_iter=n,
scoring=score,
cv=5,
scheduler=client,
n_jobs=-1
)
random_search_cv.fit(x_train, y_train)
```
And I wonder if there is a parameter like `release_job=True` to set so that we get the results returned incrementally instead of waiting for the entire cross validation to finish
Or rather retrieving the results internally and not having to recompute. By that I mean if a job finished it the parameter combination will be stored internally to the RandomizedSearchCV object so that it don't need to recompute if this worker got killed.
By doing we can efficiently prevent working being killed/restarted so that we save time waiting for the CV to finish.
Thanks
Contributor guide
Assessment
This issue has not been assessed yet.