EpistasisLab / EpistasisLab/tpot

Many cores systems problem (1 core loaded and 95 are free) and fitted_pipeline_ problem

Open
#1,349 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Hi!
Thank you for creating TPOT!

I'm using TPOT==0.12.2.

My server has 96 cores and I always notice that TPOT uses all cores only for first 5-7 minutes of each "generation" (right?). And after that TPOT uses only 1 core for next few hours.

I tried to limit that process life time using different combinations of arguments:

- generations = None
- max_time_mins = 30
- max_eval_time_mins = 30

For the first time I use to think that "max_time_mins" works like "signal" in python and rises an exception to stop a bench of processes that you call "generation". But it doesn't. And I don't understand why.

Could you tell me what combination of arguments should I set to stop one "generation" in 30 minutes?

P.S.
I've created a custom metric to explore the problem and found strange situations. For example. TPOT has found a good pipeline at the beginning of process (within first 5 minutes after start when 96 cores worked hard). Metric of that pipeline was printed and it was "perfect" for me. But after that all 95 process was finished and 1 process worked for next hour. After an hour TPOT finished all jobs and returned to fitted_pipeline_ **NOT** a "perfect" pipeline. But a "random" (?) or last(?) pipeline with very bad score. I wanted to get "perfect" pipeline (with printed score). But I received not a good pipeline. Why? If it is an error it is probably connected with that 1 super long process.

P.P.S.
It's very painful to look at one working core for few hours:

![](https://github.com/EpistasisLab/tpot/assets/30201077/14561c49-a6ec-4106-80df-0573de61915b)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.