NVIDIA-NeMo / NVIDIA-NeMo/Automodel

workaround required for padding_mask

Open
#1,639 3 comments 0 reactions 1 assignee View on GitHub

@zeel2104 is already working on this.

Since Mar 31, 2026.

bug good first issue
Dominant language
Python
Stars
963
Forks
318
Avg merge
3d 20h
Merged PRs (30d)
143

Description

Describe the bug

In particular for https://github.com/NVIDIA-NeMo/Automodel/pull/1631, after fixing the tied weights issue, I run into another issue where the default collator was creating a "padding_mask" although the model is unable to consume it, thus crashing the run.

1631 included a filtering step to remove the padding_mask, but it would be better not to have to do any filtering, as it pollutes/complicates the code.

Steps/Code to reproduce bug
Run the yaml that I have attached to 1631

Expected behavior
Run without crashing / having to filter the batch.

Additional context

Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.