alteryx / alteryx/evalml

AutoML search: start_iteration_callback can be called more times than max_iterations

Ouverte
#2,283 0 commentaires 0 réactions 1 personne assignée Réclamée par @asniyaz Voir sur GitHub
bug
Langage dominant
Python
Étoiles
850
Forks
96
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

**Problem**
If you use the `max_iterations` stopping criterion and set it to less than the size of the first batch, the `start_iteration_callback` will still get called for every pipeline in the first batch, even if some of those pipelines wouldn't be run.

**Repro**
Define start iteration callback to print something, run automl with max_iterations below the first batch size (which is currently 9), so let's say 5, and observe that the callback prints out 9 times, instead of printing out 5 times.

**Fix**
In the case where the remaining number of pipelines to search is less than the size of the next batch: we could update the `for` loop where we call `start_iteration_callback` in `AutoMLSearch.search` to iterate only over the remaining pipelines.

Tracking as known bug. I hope this will become irrelevant with the automl algorithm updates we have planned, because we'll do away with the concept of `max_iterations` entirely.

Thanks @ColinRTaylor for sharing this with us.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.