AutoML search: start_iteration_callback can be called more times than max_iterations
- 主要语言
- Python
- 星标
- 850
- 派生
- 96
- PR 合并指标
- 30 天内没有已合并 PR
描述
**Problem**
If you use the `max_iterations` stopping criterion and set it to less than the size of the first batch, the `start_iteration_callback` will still get called for every pipeline in the first batch, even if some of those pipelines wouldn't be run.
**Repro**
Define start iteration callback to print something, run automl with max_iterations below the first batch size (which is currently 9), so let's say 5, and observe that the callback prints out 9 times, instead of printing out 5 times.
**Fix**
In the case where the remaining number of pipelines to search is less than the size of the next batch: we could update the `for` loop where we call `start_iteration_callback` in `AutoMLSearch.search` to iterate only over the remaining pipelines.
Tracking as known bug. I hope this will become irrelevant with the automl algorithm updates we have planned, because we'll do away with the concept of `max_iterations` entirely.
Thanks @ColinRTaylor for sharing this with us.
贡献指南
评估
这个 Issue 还没有评估数据。