Nemo-rl OOM issue for 1.5b sft training
Open
research
t-mcore
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
When using nemo-rl for SFT on the Qwen-1.5B model, our configuration is as follows:
```
f'++policy.megatron_cfg.tensor_model_parallel_size=1 '
f'++policy.megatron_cfg.context_parallel_size=2 '
f'++policy.train_global_batch_size=1024 '
f'++policy.max_total_sequence_length=32768'
f'++policy.train_micro_batch_size=1
```
However, with these settings, we encounter an OOM (out of memory) error even when running on 64 nodes.
Interestingly, the exact same configuration works without issue when using the nemo-aligner framework.
If we change
`f'++policy.megatron_cfg.context_parallel_size=4'`
the problem disappears.
Contributor guide
Assessment
This issue has not been assessed yet.