Slime + Megatron backend: training frequently hangs after SGLang CUDA graph capture & server startup (Uvicorn ready), no further progress

Description

When running Slime RL training with megatron as the training backend, the job frequently hangs right after SGLang finishes CUDA graph capture and the OpenAI-compatible server becomes ready (Uvicorn running ...). After this point, there is no further log output and training does not progress.

This seems to be a deadlock/stall between the rollout engine and the training process (possibly engine synchronization or weight-sync step), but I cannot pinpoint the root cause yet.

Environment

• OS: Ubuntu 22.04.2 LTS (Jammy)
• GPUs: 8x H800(140G)
• Training framework: Slime
• Training backend: megatron
• Rollout engine: sglang
• Model: Qwen3-VL-2B-Instruct
• Dataset: chenhegu/geo3k_imgurl
• Launch: ./slime/examples/geo3k_vlm/run_geo3k_vlm.sh

Reproduction steps

1. Use the provided Slime geo3k VLM RL script (megatron backend).
2. Launch with Ray + colocate enabled (default in the script).
3. Wait for SGLang engines to start.

Expected behavior

After SGLang server startup, rollout requests should begin and training should continue (sampling, rewards, optimizer steps, etc.).

Actual behavior

The job hangs after SGLang prints that the server is ready. No further progress or logs from training/rollout.

Logs (hang point)

The process consistently gets stuck after the following logs:

(SGLangEngine pid=96823) 
Capturing batches (bs=4 avail_mem=53.42 GB):  33%|███▎      | 2/6 [00:01<00:01,  2.39it/s]
Capturing batches (bs=2 avail_mem=53.41 GB):  33%|███▎      | 2/6 [00:01<00:01,  2.39it/s]
Capturing batches (bs=2 avail_mem=53.41 GB):  83%|████████▎ | 5/6 [00:01<00:00,  6.54it/s]
Capturing batches (bs=1 avail_mem=53.41 GB):  83%|████████▎ | 5/6 [00:01<00:00,  6.54it/s]
(SGLangEngine pid=96824) [2026-01-23 15:16:46] Capture cuda graph end. Time elapsed: 1.64 s. mem usage=0.11 GB. avail mem=53.41 GB.
(SGLangEngine pid=96824) [2026-01-23 15:16:46] max_total_num_tokens=740481, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=2048, context_len=262144, available_gpu_mem=53.41 GB
(SGLangEngine pid=96824) [2026-01-23 15:16:47] INFO:     Started server process [97580]
(SGLangEngine pid=96824) [2026-01-23 15:16:47] INFO:     Waiting for application startup.
(SGLangEngine pid=96824) [2026-01-23 15:16:47] Using default chat sampling params from model generation config: {'repetition_penalty': 1.0, 'temperature': 0.7, 'top_k': 20, 'top_p': 0.8}
(SGLangEngine pid=96826) 
(SGLangEngine pid=96827) 
(SGLangEngine pid=96824) [2026-01-23 15:16:47] The server is fired up and ready to roll!
(SGLangEngine pid=96824) [2026-01-23 15:16:47] INFO:     Application startup complete.
(SGLangEngine pid=96824) [2026-01-23 15:16:47] INFO:     Uvicorn running on http://11.48.143.134:15050 (Press CTRL+C to quit)

Start by running ./slime/examples/geo3k_vlm/run_geo3k_vlm.sh with the documented megatron, Ray, colocate, and SGLang setup, then inspect logs around CUDA graph capture and Uvicorn startup. Trace whether rollout requests or the training/weight-sync step begins after the server-ready message; done means training proceeds through sampling, rewards, and optimizer steps without hanging.