dotnet / dotnet/machinelearning

AutoML 2.0: Should Seed be a tunable param?

Open
#6,704 2 comments 0 reactions 0 assignees View on GitHub
area-AutoML enhancement untriaged
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

One of the tunable parameters is FeatureFraction. Per my understanding this drops randomly the specified portion of columns for each tree. Further, I understand that random seed is used to decide which columns to drop.

In small forests with few trees the specific feature columns that are dropped may have bigger impact than how many features are dropped. One of the columns may lead to overfit and dropping it may improve the results on validation and test sets. However, it is crucial that the correct columns is dropped.

If my understanding is correct, then I believe seed should be one of the tunable parameters. This should be the case at least for algorithms where FeatureFraction is present (FastTree and LightGBM at least). At the moment, it is not present in e.g. FastTreeOption class.

Agree/disagree?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.