[FEATURE] support model.from_pretrained without the need of init distributed
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 270
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
from model_center.layer import CPM1
CPM1.from_pretrained("cpm1-large")
currently could not work since the function check_web_and_convert_path calls bmt.rank() or bmt.print_rank() to prevent every process downloads the checkpoint in a multi-gpu scenario.
While ModelCenter is mainly designed to support distributed training, I think it is still important to support such a common code snippet.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at CPM1.from_pretrained and trace the check_web_and_convert_path call that uses bmt.rank() or bmt.print_rank(). Verify that loading a pretrained model works without distributed initialization while retaining the multi-GPU checkpoint-download behavior. Done means the shown standalone snippet succeeds without breaking distributed use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100