AnswerDotAI / AnswerDotAI/fsdp_qlora
How to load the saved model?
- Dominant language
- Jupyter Notebook
- Stars
- 1.6k
- Forks
- 201
- PR merge metrics
- No merged PRs in 30d
Description
Hi
I need your help with loading the model. I see how you're doing that in the "converting..." file. But this is only for LORA models.
What about full_shard models? (--sharding_strategy full_shard --train_type full).
I tried to load it this way but it didn't work:
```
model = AutoModelForCausalLM.from_pretrained(model_name).to("cuda")
model.load_state_dict(torch.load('model_state_dict.safetensors'))
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the repository's "converting..." file referenced in the issue and compare its LoRA loading flow with the full_shard/full-training artifacts. Reproduce the shown AutoModelForCausalLM and load_state_dict attempt; done means documenting a verified way to load full_shard outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100