CopilotKit / CopilotKit/outpost
Worker: per-type claiming serializes job types, and per-type limits oversubscribe maxConcurrency
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7
- Forks
- 3
- Avg merge
- 7d 16h
- Merged PRs (30d)
- 15
Description
Two coupled scheduling defects in packages/outpost/queue/src/worker.ts. Distinct from #138, which covers shutdown and health reporting.
1. Per-type batches are awaited sequentially
claimAndProcessJobsPerType awaits each type's batch inside the loop, so the per-type concurrency pools never actually run concurrently. A slow type blocks every type after it in the same poll: AI_RESPONSE has a 120s timeout, so an ESCALATION or SLACK_MIRROR job queued behind it waits for the AI pipeline even though its own pool is idle.
2. The per-type limits sum above the global cap
The registered per-type limits sum to 13 while maxConcurrency is 10. Combined with the sequential claiming above, types declared last in the map are the ones starved — SLACK_MIRROR is currently last.
Either dispatch the per-type batches concurrently (Promise.all over the types, each still respecting its own limit), or make the global cap the authority and document the relationship. Whichever is chosen, the invariant deserves a test: today queue.test.ts:301 computes maxSeen and never asserts it, so a concurrency regression cannot fail the suite.
Found by: CR round 2 on #150 (slots 1-2, 2-1, 5-1).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/outpost/queue/src/worker.ts at claimAndProcessJobsPerType and trace how each type's batch is awaited and limited. Then read queue.test.ts around line 301, run the queue tests, and turn the existing maxSeen observation into assertions covering concurrent type processing and the global maxConcurrency invariant. Done means the chosen scheduling relationship is explicit and the regression tests fail when either invariant is broken.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100