EpistasisLab / EpistasisLab/tpot

Number of simultaneously running jobs fluctuates

Open
#1,087 1 comment 0 reactions 0 assignees View on GitHub
enhancement need contributor
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Hello,
I noticed that when I am running TPOT, with njobs=8 for example, and checking the processes using *top* in linux, I see 8 different processes running. However after a while that number starts decreasing until only 1 is remaining and finally, after that process has completed, 8 new processes start again and the cycle repeats.

Since the last pipeline remaining can take significantly longer than the remaining pipelines (or even cross the maximum minutes threshold), this doesn't seem to be efficient.

To me it seems that TPOT isn't starting another process until all of the initial 8 different processes have been completed. Is this the case?

Thanks !

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.