AnswerDotAI / AnswerDotAI/fsdp_qlora

How to load the saved model?

Open
#48 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.6k
Forks
201
PR merge metrics
No merged PRs in 30d

Description

Hi

I need your help with loading the model. I see how you're doing that in the "converting..." file. But this is only for LORA models.
What about full_shard models? (--sharding_strategy full_shard --train_type full).

I tried to load it this way but it didn't work:

```
model = AutoModelForCausalLM.from_pretrained(model_name).to("cuda")
model.load_state_dict(torch.load('model_state_dict.safetensors'))
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the repository's "converting..." file referenced in the issue and compare its LoRA loading flow with the full_shard/full-training artifacts. Reproduce the shown AutoModelForCausalLM and load_state_dict attempt; done means documenting a verified way to load full_shard outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.