NVIDIA-NeMo / NVIDIA-NeMo/Automodel
Unify PEFT export naming and require real HF reload tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 960
- Forks
- 316
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 143
Description
PEFT export failures keep recurring because wrapper prefixes, model-specific namespace conversion, adapter tensor layouts, and target metadata are handled in separate paths. AutoModel save/reload round trips can agree with each other while the exported artifact fails to load into the actual HF PEFT model.
Examples include AMINT-330 (Nemotron backbone/model namespace mismatch), NVIDIA-NeMo/Automodel#3656 (Omni wrapper skips conversion for PEFT-prefixed keys), NVIDIA-NeMo/Automodel#3595 (thinker prefix order), and NVIDIA-NeMo/Automodel#3592 (expert ParamWrapper layout compatibility). The narrow AMINT-330 fix is PR #3866; this follow-up stays in Backlog and does not block that fix.
Scope
- Extend the existing tiny-model HF PEFT reload/merge tests into a shared suite for supported export formats. Establish coverage before changing the common export boundary.
- Give the shared checkpoint boundary sole ownership of the PEFT outer prefix, so model adapters convert model paths without independently adding or stripping base_model.model. Apply the same contract to bulk and streaming export, and preserve loading of existing supported checkpoints.
- Keep architecture-specific renaming, multimodal nesting, and expert tensor conversion in the owning model adapter. Use the same model-owned naming rules for exported tensor keys and target_modules/target_parameters metadata.
- Make the receiving HF implementation and supported PEFT layout explicit through the existing export compatibility choices. Avoid inferring the PEFT target namespace solely from the original base checkpoint's serialized names.
- Add appropriate checks for duplicate output keys and unresolved targets. Keep exceptions for unsupported components explicit and narrow; do not load a full second model on every production save.
Acceptance criteria
- Representative tiny dense, MoE, and multimodal models export through the real metadata and tensor save path and load with the actual HF PEFT consumer.
- Tests initialize nonzero adapter weights and check every expected tensor's presence, shape, and value, plus forward or merged-weight equivalence against the native adapter behavior.
- Bulk and streaming exports agree. Explicit legacy exports and loading existing supported adapter checkpoints retain coverage.
- The suite covers supported Transformers/PEFT compatibility combinations and runs for relevant checkpoint/model changes and dependency upgrades. A skipped compatibility test is not counted as validated support.
- Existing model adapters are migrated incrementally under these checks; family-specific mappings remain model-owned. Distributed checkpoint coverage remains separate for rank gathering and TP/PP/EP behavior.
No broad refactor is included in PR #3866. The first deliverable here is the shared consumer-level regression coverage, followed by the smallest common-boundary changes that remove duplicated PEFT prefix handling.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating and running the existing tiny-model HF PEFT reload/merge tests, then trace the common export boundary and its bulk and streaming save paths. Compare those paths with the model-owned naming and metadata conversion rules described in the issue. Done means representative dense, MoE, and multimodal exports reload through the actual HF PEFT consumer with tensor, metadata, equivalence, legacy, and compatibility coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100