Early Stopping for GridSearchCV, RandomizedSearchCV

Describe the workflow you want to enable

• I have a custom model implementing the BaseEstimator, for which I am using scikit-learn's hyperparameter searches.
• I am running an exhaustive grid search, all possible parameters for my model.
• If one parameter setting gives a sufficiently high accuracy (e.g. F1 = 1.0), I wish to stop the grid search and free the compute for other runs.

Describe your proposed solution

GridSearchCV, RandomizedSearchCV, and others should have an early stopping criteria. I should be able to specify a threshold accuracy, such that when the value returned by the scoring function passes this threshold, other jobs are stopped.

Describe alternatives you've considered, if relevant

No response

Additional context

• As parameter search happens in parallel, it may require some rewrite of the parallelization code so that results can be checked as they return: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/model_selection/_search.py#L822
  See, for instance, https://github.com/joblib/joblib/issues/356#issuecomment-224508490
• I am opening this feature request to start a discussion; I am happy to do the development and submit a pull request if we agree on what should be built.

Start in sklearn/model_selection/_search.py around the parallel search code at line 822, focusing on how GridSearchCV and RandomizedSearchCV collect results. Review the linked joblib discussion before deciding how scores should be checked as jobs return. Done means an agreed early-stopping threshold can stop remaining search work without breaking parallel execution.