NVIDIA-NeMo / NVIDIA-NeMo/Automodel
Refactoring for custom models
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 963
- Forks
- 318
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 143
Description
things to consider for future refactoring:
- user should be able to use
from_pretrainedas_target_ofNeMoAutoModelForCausalLMclass, not be constrained by the specific class. is_meta_devicewas meant to be temporary, and IIRC @adil-a had the AI to remove it once an issue was resolved- I'd like to remove the backend altogether. If the user wants to use the vanilla implementation, they can use
_target_: AutoModelForCausalLM.from_pretrained; if they want to use the optimized implementation, then they can use_target_: NeMoAutoModelForCausalLM. I'm ok with adding one option toNeMoAutoModelForCausalLMto switch to the baseline implementation, but by default it should return the optimized one. enable_hf_state_dict_adapterI expect users not to know when to use this, and therefore, I feel it would be better to consume this technical debt inside automodel's code.
Originally posted by @akoumpa in https://github.com/NVIDIA-NeMo/Automodel/pull/493#discussion_r2373517008
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the NeMoAutoModelForCausalLM entry point and the from_pretrained, is_meta_device, backend, and enable_hf_state_dict_adapter behavior described here. Review the discussion in PR #493 first; the work is done only when the four proposed refactoring concerns have a decided scope and are implemented consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100