Qwen2.5-32B w/ sequence_length=16384 OOM on 16nodes H100 using FSDP2 backend
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
https://github.com/NVIDIA-NeMo/RL/commits/v0.2.1:
OOM with following script, not OOM if `policy.dynamic_batching.enabled=false`
3e5481fd7ac902e66cc0f7153957d5b443926c38 and main branch also:
OOM with both `policy.dynamic_batching.enabled=true` and `=false`
Repro:
```bash
RUN_COMMAND="uv run python examples/run_grpo_math.py \
--config examples/configs/grpo_math_8B.yaml \
policy.model_name="Qwen/Qwen2.5-32B" \
policy.generation.vllm_cfg.tensor_parallel_size=4 \
policy.max_total_sequence_length=16384 \
policy.dtensor_cfg.enabled=true \
policy.dtensor_cfg.tensor_parallel_size=8 \
policy.dtensor_cfg.sequence_parallel=true \
policy.dtensor_cfg.activation_checkpointing=true \
policy.dynamic_batching.enabled=true \
policy.dynamic_batching.train_mb_tokens=16384 \
policy.dynamic_batching.logprob_mb_tokens=32768 \
checkpointing.enabled=false \
logger.wandb_enabled=true \
logger.tensorboard_enabled=false \
logger.monitor_gpus=true \
logger.wandb.project=${PROJECT_NAME} \
logger.wandb.name=${EXP_NAME} \
cluster.num_nodes=16 \
cluster.gpus_per_node=8"
```
Contributor guide
Assessment
This issue has not been assessed yet.