NVIDIA-NeMo / NVIDIA-NeMo/RL

Why the rows in train_data should be flattened within the from_batches function?

Open
#1,077 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

**Describe the bug**

When running GRPO, the function in `nemo_rl\distributed\batched_data_dict.py` lines 135-137 flattens each row of the tensors in `train_data`. However, the rows passed here are all 1D (with a `seq_len` dimension), it seems flattening has no effect..

When I passed full logits with shape `(batch_size, seq_len, vocab_size)` and enabling sequence packing, this flattening incorrectly merges the last two dimensions (`seq_len` and `vocab_size`), leading to bugs. Why we need the rows flattened within the from_batches function?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.