EpistasisLab / EpistasisLab/tpot

Slow/Freeze with powerful configuration

Open
#905 6 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Hi there,

With the cluster of my university, I'm trying to run TPOT for a classification problem on a jupyter notebook with a datset of size (166'158x193). I cannot provide the dataset for confidentialy reasons.

Despite:
- powerful instance (36 cores, 180 GB RAM)
- tuning parameters ("TPOT LIGHT", reducing generation and population_size)
- consulting other issues (reinstall TPOT development branch,...)
- using Dask

The fitting time is vers slow and the progress bar only progress if i stop the kernel:
![Capture d’écran 2019-08-17 à 15 33 05](https://user-images.githubusercontent.com/40396043/63212403-5913a380-c104-11e9-8231-82332c58342c.jpg)

Here is my configuration:
![Capture d’écran 2019-08-17 à 15 34 15](https://user-images.githubusercontent.com/40396043/63212424-b27bd280-c104-11e9-8aa1-b648f3e9d772.jpg)

Here is my dask cluster:
![Capture d’écran 2019-08-17 à 15 35 04](https://user-images.githubusercontent.com/40396043/63212430-c7586600-c104-11e9-84e1-afd8438c5c84.jpg)

Here is my TPOT code:
```python
pipeline_optimizer = TPOTClassifier(generations=20,
population_size=50,
scoring='neg_log_loss',
cv=ut.RepeatedHoldout(n_iter=30, train_size=0.6, test_size=0.1),
random_state=42,
verbosity=2,
n_jobs=-1,
warm_start = True,
use_dask=True)
pipeline_optimizer.fit(df_cv[xvars].values, df_cv[yvars].values, weight)
```

where `ut.RepeatedHoldout()` is a personal cross-validation generator to deal with Time-dependant data
So I'm wondering if missed any important parameters to run smoothly TPOT,...

Thank you for any help

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.