Question about saving peft checkpoint
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🐛 Describe the bug
From my understand, when saving checkpoints for peft models (see [here](https://github.com/CarperAI/trlx/blob/bcd237f1e94c84c5c9f5a4086bab34c0946e3fa7/trlx/trainer/accelerate_base_trainer.py#L315)), trlx removes `pytorch_model.bin` before calling `save_pretrained` which makes the removal useless in my opinion.
Is this intentional or we should move the removal code after `save_pretrained` is called?
Here is an example of a directory resulting from `save_pretrained`:
```
adapter_config.json adapter_model.bin optimizer.bin pytorch_model.bin random_states_0.pkl special_tokens_map.json spiece.model tokenizer_config.json tokenizer.json
```
### Which trlX version are you using?
0.7.0
### Additional system and package information
3.10.12
Contributor guide
Research direction
Start in trlx/trainer/accelerate_base_trainer.py around line 315 and trace the checkpoint-saving order for PEFT models. Reproduce the save_pretrained output with trlX 0.7.0, then verify whether pytorch_model.bin is removed at the intended point and whether the resulting checkpoint contents match the expected behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100