sindresorhus / sindresorhus/p-queue

`.add()` does not check `isPaused`, queue can grow unbounded after `.pause()`

Open
#253 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
4.3k
Forks
220
PR merge metrics
No merged PRs in 30d

Description

.pause() stops the queue from running new tasks. It does not stop .add() from accepting more work.

In our system several independent call sites call queue.add(). HTTP handlers, background pollers, etc. They don't know about each other.

When we pause() the queue intending to stop it, .add() keeps succeeding from every one of those call sites. queue.size grows without bound. This caused a real memory leak for us.

Minimal repro:

import PQueue from 'p-queue';

const queue = new PQueue({concurrency: 1});
queue.pause();

for (let i = 0; i < 1_000_000; i++) {
      queue.add(async () => {}); // succeeds every time
}

console.log(queue.size); // 1000000

queue.isPaused is already public. So in theory every call site could check it before calling add().

In practice that means every call site has to remember to do this manually. It's easy to miss one. That's what happened here.

It's also not atomic with the .add() call. pause() can run on another code path between the check and the .add() call. So even a careful caller can still queue a task after the intent was to stop.

We saw #239. We understand clear()'s dangling-promise behavior is intentional, and that AbortSignal is the recommended cancellation primitive.

We don't think AbortSignal solves this case though. It cancels tasks that were already added. It doesn't stop a producer from calling .add() again in the first place.

Is this the expected behavior? Is there a recommended pattern for stopping the queue from accepting new work, one that doesn't require every producer to remember to check isPaused themselves? Happy to send a small, additive PR if a guard along these lines would be welcome.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing pause(), isPaused, and add() behavior in p-queue, then review issue #239 for the intended cancellation semantics. Determine and document the accepted behavior when add() is called after pause(), including how callers and returned promises should be handled. Add focused tests for paused-queue submissions and verify the full test suite.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.