Tencent / Tencent/AngelSpec

SGLang multi-node: auto-negotiated dist_init port can collide with pre-allocated engine ports

Open
#7 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
270
Forks
28
PR merge metrics
No merged PRs in 30d

Description

In _prepare_sgl_engines (angelspec/inference/factory.py), Step 2 picks a free port for dist_init_addr starting from the default start_port=10000:

https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L288-L295

ip, port = ray.get(
    [head_engine.get_node_ip.remote(), head_engine.find_free_port.remote()],
    timeout=120,
)

find_free_port only probes ports; it never reserves them. Step 2.5 then restarts its sequential allocation at the literal next_start = 10000 without accounting for the port just handed out:

https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L297-L309

next_start = 10000
for i in range(num_engines):
    port = ray.get(engines[i].find_free_port.remote(start_port=next_start, consecutive=2), ...)
    pre_allocated_ports[i] = port
    next_start = port + 2

On a fresh node, Step 2 returns 10000 and Step 2.5 immediately hands engine 0 (which is the replica-0 head on the same node) ports 10000/10001. SglEngine.init then sets port=10000, nccl_port=10001, and dist_init_addr="<same-ip>:10000" — the head node is told to bind its server port and its dist-init TCPStore on the same port. The Step 2.5 comment says "allocate sequentially so engines on the same node never collide", but the Step 2 allocation is invisible to that sequencer.

The vLLM path avoids this by reusing the head's pre-allocated port as the dist-init port rather than drawing a second untracked port. Possible fixes: allocate the dist-init port from the same sequential allocator (e.g. run Step 2 after Step 2.5 using next_start, or advance next_start past the negotiated dist-init port). Only affects sglang_nnodes > 1 with auto-negotiated addr.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in angelspec/inference/factory.py at _prepare_sgl_engines, reading Step 2 and Step 2.5 together, then trace SglEngine.init to confirm how port and dist_init_addr are assigned. Ensure the auto-negotiated dist-init port cannot overlap pre-allocated engine ports in the multi-node SGLang path, especially when the default starts at 10000.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems, networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.