andywer / andywer/threads.js

expose `size` in `spawnWorker` callback

Open
#354 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
3.5k
Forks
173
PR merge metrics
No merged PRs in 30d

Description

I'm trying to send the thread count to every worker. There are certain timeouts, like [file locks](https://www.npmjs.com/package/lockfile), used for allowing only one thread in at a time. I have the lock wait timeout set to a multiplier based on how many threads are running.

I was trying to do this:

```js
let threads;

function spawnWorker() {
return spawn(new Worker('foo', {
env: {
...process.env,

// useful for calculating timeouts based on how many other tasks are running
THREADS: threads,
},
}));
}

let pool = Pool(spawnWorker);

threads = pool.workers.length;
```

but since the initial pool spawning is synchronous, the env var gets hit before the `threads = pool.workers.length;` line.

My work around is of course:

```js
THREADS: require('os').cpus().length,
```

but I would prefer to be directly tied to your value so they are always in sync.

My proposition is the send the pool size down to the `spawnWorker` callback.

This:

```js
function spawnWorkers(spawnWorker, count) {
return createArray(count).map(() => ({
init: spawnWorker(),
runningTasks: []
}));
}
```

becomes

```js
function spawnWorkers(spawnWorker, count) {
return createArray(count).map(() => ({
init: spawnWorker(count),
runningTasks: []
}));
}
```

and my code becomes

```js
function spawnWorker(size) {
return spawn(new Worker('foo', {
env: {
...process.env,

// useful for calculating timeouts based on how many other tasks are running
THREADS: size,
},
}));
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.