[Question] Stuck at "Dummy health check server started" when training DeepSeek-R1
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
I'm trying to train DeepSeek-R1 using the training script you provided, but the process gets stuck at:
Capturing batches (bs=8 avail_mem=35.24 GB): 100%|██████████| 48/48 [03:38<00:00, 4.55s/it]
(SGLangEngine pid=1279612, ip=10.1.117.214) [2025-12-06 13:02:52 DP14 TP117 EP7] Capture cuda graph end. Time elapsed: 219.21 s. mem usage=5.14 GB. avail mem=35.10 GB.
(SGLangEngine pid=1276837, ip=10.1.3.172) [2025-12-06 13:02:52 DP0 TP0 EP0] max_total_num_tokens=1234176, chunked_prefill_size=512, max_prefill_tokens=16384, max_running_requests=4096, context_len=4096, available_gpu_mem=35.18 GB
(SGLangEngine pid=1282393, ip=10.1.24.84) [2025-12-06 13:02:53] Dummy health check server started in background thread at 10.1.24.84:15000
So I have a couple of questions:
-
Does slime officially support training large MoE models such as DeepSeek-R1?
If so, are there specific flags or rollout configurations required for large MoE models? -
What should I expect after launching ?
- Should I wait for additional logs ?
- Does the script automatically trigger rollout after the health check message?
- Is the delay normal for large MoE models, or is it indicative of a misconfiguration?
Any clarification would be appreciated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report only identifies the provided training script and startup logs, ending at the SGLang health-check message. Start by reviewing the training script's DeepSeek-R1 and rollout configuration, then determine whether rollout begins after this message and whether the delay is expected; done means documenting the cause and required configuration or confirming a reproducible startup failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100