alteryx / alteryx/evalml

AutoML search: start_iteration_callback can be called more times than max_iterations

未关闭
#2,283 0 条评论 0 个 reaction 已指派 1 人 已被 @asniyaz 认领 在 GitHub 查看
bug
主要语言
Python
星标
850
派生
96
PR 合并指标
30 天内没有已合并 PR

描述

**Problem**
If you use the `max_iterations` stopping criterion and set it to less than the size of the first batch, the `start_iteration_callback` will still get called for every pipeline in the first batch, even if some of those pipelines wouldn't be run.

**Repro**
Define start iteration callback to print something, run automl with max_iterations below the first batch size (which is currently 9), so let's say 5, and observe that the callback prints out 9 times, instead of printing out 5 times.

**Fix**
In the case where the remaining number of pipelines to search is less than the size of the next batch: we could update the `for` loop where we call `start_iteration_callback` in `AutoMLSearch.search` to iterate only over the remaining pipelines.

Tracking as known bug. I hope this will become irrelevant with the automl algorithm updates we have planned, because we'll do away with the concept of `max_iterations` entirely.

Thanks @ColinRTaylor for sharing this with us.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。