futureverse / futureverse/future

Creating and customizing n heterogeneous clusters (or workers)

Open
#389 4 comments 0 reactions 0 assignees View on GitHub
pkg/parallelly
Dominant language
R
Stars
1k
Forks
92
PR merge metrics
No merged PRs in 30d

Description

Hello,
I am new to "future" terminology so please forgive me, I may be confusing when I use process/core/thread/workers etc...
Basically, I am interested in parallel processing on a single multi-core CPU.
Like to know if it is possible to create, let say, n clusters but each with a different number of threads/cores.
For example, create 4 clusters in parallel in this way:

cl1 with 8 cores
cl2 with 5 cores
cl3 with 4 cores
cl4 with 7 cores

Can "cl" be vector or list, like cl[i] or cl[[i]]?
The number of cores/threads will be in an array/list/vector. So I will need to give
it to the process/worker creation as a parameter or set the related option for it.

I would be nice to run them in a parallel loop like foreach .. %dopar% {}

So a total of 24 cores in parallel will work but grouped for 4 different tasks.
Do you know a way to do it?
Please examples.

Thanks...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.