alteryx / alteryx/evalml

AutoML search: start_iteration_callback can be called more times than max_iterations

オープン
#2,283 コメント 0 件 リアクション 0 件 担当者 1 名 @asniyaz が担当を希望しています GitHub で見る
bug
主要言語
Python
スター
850
フォーク
96
PR マージ指標
30日以内にマージされた PR はありません

説明

**Problem**
If you use the `max_iterations` stopping criterion and set it to less than the size of the first batch, the `start_iteration_callback` will still get called for every pipeline in the first batch, even if some of those pipelines wouldn't be run.

**Repro**
Define start iteration callback to print something, run automl with max_iterations below the first batch size (which is currently 9), so let's say 5, and observe that the callback prints out 9 times, instead of printing out 5 times.

**Fix**
In the case where the remaining number of pipelines to search is less than the size of the next batch: we could update the `for` loop where we call `start_iteration_callback` in `AutoMLSearch.search` to iterate only over the remaining pipelines.

Tracking as known bug. I hope this will become irrelevant with the automl algorithm updates we have planned, because we'll do away with the concept of `max_iterations` entirely.

Thanks @ColinRTaylor for sharing this with us.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。