1jehuang / 1jehuang/jcode

fill_slots assigns the same node multiple times to one worker

Open
#1,209 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-info
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Symptom

When calling fill_slots with concurrency_limit > 1 over a ready frontier wider than 1, the same node is assigned multiple times to the same session. Observed: 8 identical assignments of catalog-evaluate to session_kangaroo over a 13-wide frontier.

Repro

  1. Seed a plan with multiple ready nodes (e.g. 13-wide frontier).
  2. Call fill_slots with concurrency_limit=10.
  3. Observe the assignment log: one node appears many times for one session.

Root cause

The fill loop in communicate.rs does not deduplicate by node — each slot-fill iteration re-picks the head-of-queue node when the ready set has not been consumed yet.

Proposed fix

Deduplicate per-node within the fill loop and spread assignments across distinct ready nodes first. A fix implementing this is available on the fork (dwillitzer/jcode, commit 7a24c6917, "fix(swarm): spread fill slots across ready nodes").

Cross-program evidence

Independently reproduced by a separate fleet with its own coordinator: assignments without explicit target mis-route, same dispatch-path root (ledger "Cross-Program Evidence (Synapse Fleet)", commit 7b03c298c, Finding C).

Environment

Observed on jcode v0.81.x with a deep task graph and multiple ready nodes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in communicate.rs at the fill_slots loop and reproduce the issue with a 13-wide ready frontier and concurrency_limit=10. Compare the assignment log against the ready nodes and the proposed fork commit 7a24c6917. Done means one worker session receives distinct ready nodes before any node is reused, with the duplicate assignment no longer occurring.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.