Upstream SGLang fix for disaggregated prefill TP-rank desync / 向 SGLang 上游提交分离式预填充 TP rank 不同步修复

Open
#2,151 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
distributed-systems, machine-learning

Research direction

Start with SGLang's resolve_waiting_queue_bootstrap() and the local workaround from #2127. Reproduce or inspect how candidate sets differ across TP ranks, then compare the behavior with the non-aborted pending_bootstrap restriction. Done means an upstream SGLang fix prevents ranks from entering or skipping the collective inconsistently and supersedes the local workaround.

Written by the indexing model from the issue text.

Description

Track an upstream SGLang fix for the local patch added in #2127.

resolve_waiting_queue_bootstrap() can build different candidate sets across TP ranks, causing some ranks to enter the collective while others skip it. The local workaround limits candidates to non-aborted pending_bootstrap requests.

Reference: https://github.com/SemiAnalysisAI/InferenceX/pull/2127#discussion_r3560092434

中文说明

跟踪 #2127 中本地补丁对应的 SGLang 上游修复。resolve_waiting_queue_bootstrap() 可能在不同 TP rank 上生成不一致的候选集合，导致 collective 不同步。当前临时方案仅保留未中止的 pending_bootstrap 请求。

Dominant language
Python
Stars
1.7k
Forks
303
Avg merge
1d 13h
Merged PRs (30d)
284

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from SemiAnalysisAI/InferenceX

All issues in SemiAnalysisAI/InferenceX

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.