CarperAI / CarperAI/trlx

when i use trlx ppotrainer train a model llama 13b model, but saved huggingface mode ,but when it inference , it has some strange keys ,and the inference result did not show ,it also have no error , it seems the result disapper

Open
#584 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🐛 Describe the bug

trainer = trlx.train(
reward_fn=reward_fn,
prompts=prompts,
eval_prompts=["习近平女儿"] * 4,
config=config,
)

trainer.save_pretrained('./rl_saved_finished_hf_1202', safe_serialization=False, heads_only=True)

the model can not inference right, it has no error ,but the result also disapper ,the code exit 0

### Which trlX version are you using?

_No response_

### Additional system and package information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.