microsoft / microsoft/FLAML

Overfitting when using AutoML

Open
#1,140 3 comments 0 reactions 1 assignee Claimed by @skzhang1 View on GitHub
question
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
565
Avg merge
5d 8m
Merged PRs (30d)
17

Description

Hi,

We used FLAML to perform regression task, and found AutoML model was easy to be overfitted. However, in the same task, other ML models e.g., LightGBM, RF, could avoid overfitting by grid search best parameters. We tried add 'cv=5' into the AutoML model, but it did not work on our case.

So could you give me some suggestions on how to avoid overfitting when using FLAML AutoML models?

BTW:
We also used `flame.default.LGBMRegressor()` to perform auto-search hyper-parameters of LightGBM model, but this model is still overfitting. But LightGBM model could be avoid overfitting by grid search methods. So I think maybe I misuse FLAML.

Lu Li

---------------------------------------------------------------------------------------

The code of FLAML AutoML models:
`from flaml import AutoML`
`am = AutoML()`
`am.fit(x_train, y_train, task="regression")`

The performance on training data:
![14031690032103_ pic](https://github.com/microsoft/FLAML/assets/46986798/3829eeed-cd90-4ae8-b42e-3d04e0b099ab)

The performance on test data:
![14041690032104_ pic](https://github.com/microsoft/FLAML/assets/46986798/610cf7c8-f8bc-477d-bf8d-659c34e0403a)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.