fill_slots assigns the same node multiple times to one worker
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.9k
- Forks
- 2.3k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 30
Description
Symptom
When calling fill_slots with concurrency_limit > 1 over a ready frontier wider than 1, the same node is assigned multiple times to the same session. Observed: 8 identical assignments of catalog-evaluate to session_kangaroo over a 13-wide frontier.
Repro
- Seed a plan with multiple ready nodes (e.g. 13-wide frontier).
- Call
fill_slotswithconcurrency_limit=10. - Observe the assignment log: one node appears many times for one session.
Root cause
The fill loop in communicate.rs does not deduplicate by node — each slot-fill iteration re-picks the head-of-queue node when the ready set has not been consumed yet.
Proposed fix
Deduplicate per-node within the fill loop and spread assignments across distinct ready nodes first. A fix implementing this is available on the fork (dwillitzer/jcode, commit 7a24c6917, "fix(swarm): spread fill slots across ready nodes").
Cross-program evidence
Independently reproduced by a separate fleet with its own coordinator: assignments without explicit target mis-route, same dispatch-path root (ledger "Cross-Program Evidence (Synapse Fleet)", commit 7b03c298c, Finding C).
Environment
Observed on jcode v0.81.x with a deep task graph and multiple ready nodes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in communicate.rs at the fill_slots loop and reproduce the issue with a 13-wide ready frontier and concurrency_limit=10. Compare the assignment log against the ready nodes and the proposed fork commit 7a24c6917. Done means one worker session receives distinct ready nodes before any node is reused, with the duplicate assignment no longer occurring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100