EpistasisLab / EpistasisLab/tpot

Simple Random Forest outperforms the best pipeline from tpot. Why?

Open
#1,016 7 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I'm trying to perform a regression analysis on an annual revenue using 130 variables (127 numeric variables and 3 categorical variables that have been label encoded). These are the parameters that I feed into tpot (note: the scoring function is r2):

`"tpot_regressor": {
"generations": 100,
"population_size": 100,
"offspring_size": 100,
"scoring": "r2",
"cv": 5,
"subsample": 1.0,
"n_jobs": -2,
"random_state": 12345,
"verbosity": 3,
"periodic_checkpoint_folder": "C:/Users/wwang/PycharmProjects/Current_Lot_POI_Modeling/tpot_temp",
"memory": "auto"
}`

The best pipeline that was generated from tpot is:

`exported_pipeline = make_pipeline(
make_union(
make_union(
FunctionTransformer(copy),
StackingEstimator(estimator=make_pipeline(
StackingEstimator(estimator=ElasticNetCV(l1_ratio=0.4, tol=0.0001)),
GradientBoostingRegressor(alpha=0.9, learning_rate=0.01, loss="huber", max_depth=5, max_features=0.55, min_samples_leaf=15, min_samples_split=17, n_estimators=100, subsample=0.7000000000000001)
))
),
FunctionTransformer(copy)
),
XGBRegressor(learning_rate=0.1, max_depth=4, min_child_weight=9, n_estimators=100, nthread=1, objective="reg:squarederror", subsample=0.6000000000000001)
)`

The r2 score of the selected tpot pipeline is: 0.6166010461141351

Then I run a simple random forest regressor just for comparison, the r2 score of the random forest is: 0.625272

I wonder why a simple random forest outperforms the tpot r2 score. Am I doing something wrong?

Thanks in advance.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.