NVIDIA-NeMo / NVIDIA-NeMo/RL

[DTensor] Qwen3-30B-A3B fail/hang at Training policy step in both v1/v2

Open
#1,066 0 comments 0 reactions 1 assignee Claimed by @adil-a View on GitHub
bug t-pytdensor x-intercom
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

As title.

1. I have tried w/ 8 nodes and 16 nodes, train TP=4, both of them failed/hang at `Training policy step`.
2. I tried remove `torch.cuda.empty_cache()` mentioned in https://github.com/NVIDIA-NeMo/RL/issues/1036, but not help.

**Test Result**
https://wandb.ai/nvidia/grpo-dev-yukih-moe/runs/hogqjorw
https://wandb.ai/nvidia/grpo-dev-yukih-moe/runs/ggaqn04c

Image


**Repro:**
Commit: https://github.com/NVIDIA-NeMo/RL/commit/9301d36cbf847212430b84a27cfe6990f773b7cf

Copy `optimizer` and `scheduler` from `examples/configs/grpo_math_8B.yaml` to `examples/configs/grpo_math_qwen30ba3b_megatron.yaml`, then run the following script.

```bash
RUN_COMMAND="NRL_FORCE_REBUILD_VENVS=true uv run python examples/run_grpo_math.py \
--config examples/configs/grpo_math_qwen30ba3b_megatron.yaml \
grpo.max_num_steps=10 \
grpo.val_period=100 \
grpo.val_at_start=false \
policy.model_name=Qwen/Qwen3-30B-A3B \
policy.dtensor_cfg.enabled=true \
policy.dtensor_cfg._v2=true \
policy.dtensor_cfg.cpu_offload=false \
policy.dtensor_cfg.sequence_parallel=false \
policy.dtensor_cfg.activation_checkpointing=false \
policy.dtensor_cfg.tensor_parallel_size=4 \
policy.dtensor_cfg.context_parallel_size=1 \
policy.dtensor_cfg.custom_parallel_plan=null \
policy.megatron_cfg.enabled=false \
checkpointing.enabled=false \
logger.wandb_enabled=true \
logger.tensorboard_enabled=false \
logger.monitor_gpus=true \
logger.wandb.project=${PROJECT_NAME} \
logger.wandb.name=${EXP_NAME} \
cluster.num_nodes=${NUM_ACTOR_NODES} \
cluster.gpus_per_node=${NUM_ACTOR_GPUS}"
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.