when i use trlx ppotrainer train a model llama 13b model, but saved huggingface mode ,but when it inference , it has some strange keys ,and the inference result did not show ,it also have no error , it seems the result disapper
Open
bug
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🐛 Describe the bug
trainer = trlx.train(
reward_fn=reward_fn,
prompts=prompts,
eval_prompts=["习近平女儿"] * 4,
config=config,
)
trainer.save_pretrained('./rl_saved_finished_hf_1202', safe_serialization=False, heads_only=True)
the model can not inference right, it has no error ,but the result also disapper ,the code exit 0
### Which trlX version are you using?
_No response_
### Additional system and package information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.