SGLang multi-node: auto-negotiated dist_init port can collide with pre-allocated engine ports
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 270
- Forks
- 28
- PR merge metrics
- No merged PRs in 30d
Description
In _prepare_sgl_engines (angelspec/inference/factory.py), Step 2 picks a free port for dist_init_addr starting from the default start_port=10000:
https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L288-L295
ip, port = ray.get(
[head_engine.get_node_ip.remote(), head_engine.find_free_port.remote()],
timeout=120,
)
find_free_port only probes ports; it never reserves them. Step 2.5 then restarts its sequential allocation at the literal next_start = 10000 without accounting for the port just handed out:
https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L297-L309
next_start = 10000
for i in range(num_engines):
port = ray.get(engines[i].find_free_port.remote(start_port=next_start, consecutive=2), ...)
pre_allocated_ports[i] = port
next_start = port + 2
On a fresh node, Step 2 returns 10000 and Step 2.5 immediately hands engine 0 (which is the replica-0 head on the same node) ports 10000/10001. SglEngine.init then sets port=10000, nccl_port=10001, and dist_init_addr="<same-ip>:10000" — the head node is told to bind its server port and its dist-init TCPStore on the same port. The Step 2.5 comment says "allocate sequentially so engines on the same node never collide", but the Step 2 allocation is invisible to that sequencer.
The vLLM path avoids this by reusing the head's pre-allocated port as the dist-init port rather than drawing a second untracked port. Possible fixes: allocate the dist-init port from the same sequential allocator (e.g. run Step 2 after Step 2.5 using next_start, or advance next_start past the negotiated dist-init port). Only affects sglang_nnodes > 1 with auto-negotiated addr.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in angelspec/inference/factory.py at _prepare_sgl_engines, reading Step 2 and Step 2.5 together, then trace SglEngine.init to confirm how port and dist_init_addr are assigned. Ensure the auto-negotiated dist-init port cannot overlap pre-allocated engine ports in the multi-node SGLang path, especially when the default starts at 10000.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100