NVIDIA-NeMo / NVIDIA-NeMo/Automodel

Refactoring for custom models

Open
#501 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
963
Forks
318
Avg merge
3d 20h
Merged PRs (30d)
143

Description

things to consider for future refactoring:

  1. user should be able to use from_pretrained as _target_ of NeMoAutoModelForCausalLM class, not be constrained by the specific class.
  2. is_meta_device was meant to be temporary, and IIRC @adil-a had the AI to remove it once an issue was resolved
  3. I'd like to remove the backend altogether. If the user wants to use the vanilla implementation, they can use _target_: AutoModelForCausalLM.from_pretrained; if they want to use the optimized implementation, then they can use _target_: NeMoAutoModelForCausalLM. I'm ok with adding one option to NeMoAutoModelForCausalLM to switch to the baseline implementation, but by default it should return the optimized one.
  4. enable_hf_state_dict_adapter I expect users not to know when to use this, and therefore, I feel it would be better to consume this technical debt inside automodel's code.

Originally posted by @akoumpa in https://github.com/NVIDIA-NeMo/Automodel/pull/493#discussion_r2373517008

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the NeMoAutoModelForCausalLM entry point and the from_pretrained, is_meta_device, backend, and enable_hf_state_dict_adapter behavior described here. Review the discussion in PR #493 first; the work is done only when the four proposed refactoring concerns have a decided scope and are implemented consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python, pytorch
Domain
machine-learning
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.