EpistasisLab / EpistasisLab/tpot

Record the progress and auto disable the obviously slow candidate(never use in future trainning)

Open
#328 3 comments 2 reactions 0 assignees View on GitHub
enhancement need contributor
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

features:
X.shape (9516, 24956)
code:
```
tpot = TPOTClassifier(generations=5, population_size=500, verbosity=2, max_eval_time_mins=20)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
```

log:
```
Optimization Progress: 1%| | 24/3000 [50:06<143:35:50, 173.71s/pipeline]rogress: 0%| | 1/3000 [01:29<74:23:16, 89.30s/pipeline] Optimization Progress: 1%| | 25/3000 [50:06<397:45:25, 481.32s/pipeline]

Timeout during evaluation of pipeline #25. Skipping to the next pipeline.

Optimization Progress: 1%|▏ | 44/3000 [1:21:41<40:07:44, 48.87s/pipeline]
Optimization Progress: 1%| | 28/3000 [50:16<195:53:14, 237.28s/pipeline]
Optimization Progress: 2%|▏ | 45/3000 [1:21:41<323:34:33, 394.20s/pipeline]

Timeout during evaluation of pipeline #45. Skipping to the next pipeline.

Optimization Progress: 2%|▏ | 57/3000 [1:45:48<39:40:12, 48.53s/pipeline]
Optimization Progress: 2%|▏ | 48/3000 [1:22:15<163:14:27, 199.07s/pipeline]
Optimization Progress: 2%|▏ | 58/3000 [1:45:48<125:42:37, 153.83s/pipeline]

Timeout during evaluation of pipeline #58. Skipping to the next pipeline.

Optimization Progress: 2%|▏ | 66/3000 [1:46:12<16:09:53, 19.83s/pipeline]]
Optimization Progress: 2%|▏ | 61/3000 [1:46:05<63:41:23, 78.01s/pipeline]
```

It looks like some specifical models/ models combination are very slow, so the pipeline which contain that modle would hit timeout and failed every time. I think tpot could log the info, when failed count hit a certain number, then don't use that model or delay it .(Also can use some machine learning model to select the obviously slow candidate )

PS: The log message didn't look well in jupyter notebook:
![qq 20161215161405](https://cloud.githubusercontent.com/assets/3938751/21216397/8a330fba-c2e1-11e6-9895-5b46644c7ac2.png)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.