maybe bug in prepare & load's order
Open
bug
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🐛 Describe the bug
https://github.com/CarperAI/trlx/blob/main/trlx/trlx.py#L140
this ```load``` function may be called after ```prepare``` in
https://github.com/CarperAI/trlx/blob/main/trlx/trainer/accelerate_base_trainer.py#L86
according to
https://huggingface.co/docs/accelerate/usage_guides/checkpoint
this may cause the loading not work actually, such as deepspeed model
### Which trlX version are you using?
_No response_
### Additional system and package information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.