THUDM / THUDM/slime

[Question] Stuck at "Dummy health check server started" when training DeepSeek-R1

Open
#1,043 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.5k
Forks
1.3k
Avg merge
5h 36m
Merged PRs (30d)
22

Description

I'm trying to train DeepSeek-R1 using the training script you provided, but the process gets stuck at:

Capturing batches (bs=8 avail_mem=35.24 GB): 100%|██████████| 48/48 [03:38<00:00,  4.55s/it]
(SGLangEngine pid=1279612, ip=10.1.117.214) [2025-12-06 13:02:52 DP14 TP117 EP7] Capture cuda graph end. Time elapsed: 219.21 s. mem usage=5.14 GB. avail mem=35.10 GB.
(SGLangEngine pid=1276837, ip=10.1.3.172) [2025-12-06 13:02:52 DP0 TP0 EP0] max_total_num_tokens=1234176, chunked_prefill_size=512, max_prefill_tokens=16384, max_running_requests=4096, context_len=4096, available_gpu_mem=35.18 GB
(SGLangEngine pid=1282393, ip=10.1.24.84) [2025-12-06 13:02:53] Dummy health check server started in background thread at 10.1.24.84:15000

So I have a couple of questions:

  1. Does slime officially support training large MoE models such as DeepSeek-R1?
    If so, are there specific flags or rollout configurations required for large MoE models?

  2. What should I expect after launching ?

  • Should I wait for additional logs ?
  • Does the script automatically trigger rollout after the health check message?
  • Is the delay normal for large MoE models, or is it indicative of a misconfiguration?

Any clarification would be appreciated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report only identifies the provided training script and startup logs, ending at the SGLang health-check message. Start by reviewing the training script's DeepSeek-R1 and rollout configuration, then determine whether rollout begins after this message and whether the delay is expected; done means documenting the cause and required configuration or confirming a reproducible startup failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.