alteryx / alteryx/evalml

AutoML search: start_iteration_callback can be called more times than max_iterations

Abierto
#2,283 0 comentarios 0 reacciones 1 asignado Reclamado por @asniyaz Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
850
Forks
96
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

**Problem**
If you use the `max_iterations` stopping criterion and set it to less than the size of the first batch, the `start_iteration_callback` will still get called for every pipeline in the first batch, even if some of those pipelines wouldn't be run.

**Repro**
Define start iteration callback to print something, run automl with max_iterations below the first batch size (which is currently 9), so let's say 5, and observe that the callback prints out 9 times, instead of printing out 5 times.

**Fix**
In the case where the remaining number of pipelines to search is less than the size of the next batch: we could update the `for` loop where we call `start_iteration_callback` in `AutoMLSearch.search` to iterate only over the remaining pipelines.

Tracking as known bug. I hope this will become irrelevant with the automl algorithm updates we have planned, because we'll do away with the concept of `max_iterations` entirely.

Thanks @ColinRTaylor for sharing this with us.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.