NVIDIA-NeMo / NVIDIA-NeMo/RL

Nemo-rl OOM issue for 1.5b sft training

Open
#903 0 comments 0 reactions 0 assignees View on GitHub
research t-mcore
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

When using nemo-rl for SFT on the Qwen-1.5B model, our configuration is as follows:
```
f'++policy.megatron_cfg.tensor_model_parallel_size=1 '
f'++policy.megatron_cfg.context_parallel_size=2 '
f'++policy.train_global_batch_size=1024 '
f'++policy.max_total_sequence_length=32768'
f'++policy.train_micro_batch_size=1
```
However, with these settings, we encounter an OOM (out of memory) error even when running on 64 nodes.

Interestingly, the exact same configuration works without issue when using the nemo-aligner framework.
If we change
`f'++policy.megatron_cfg.context_parallel_size=4'`
the problem disappears.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.