dask / dask/dask-ml

Release finished model/job during cross validation

Open
#415 8 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

I'm wondering if there is a way to release the job/worker when a fitting job finishes during RandomizedSearchCV.

I'm doing something like
```
random_search_cv = RandomizedSearchCV(
estimator=model,
param_distributions=params_grid,
n_iter=n,
scoring=score,
cv=5,
scheduler=client,
n_jobs=-1
)
random_search_cv.fit(x_train, y_train)
```
And I wonder if there is a parameter like `release_job=True` to set so that we get the results returned incrementally instead of waiting for the entire cross validation to finish
Or rather retrieving the results internally and not having to recompute. By that I mean if a job finished it the parameter combination will be stored internally to the RandomizedSearchCV object so that it don't need to recompute if this worker got killed.
By doing we can efficiently prevent working being killed/restarted so that we save time waiting for the CV to finish.
Thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.