alibaba / alibaba/FederatedScope
使用helm对微调后的模型进行评估,微调后的参数没有正确加载
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 261
- PR merge metrics
- No merged PRs in 30d
Description
使用的模型是opt-2.7b,使用lora进行微调,数据集是dolly15k,在docker容器里进行评估。

使用如图的yaml文件,评价命令如下

ckpt文件没有正确加载到评估里,导致使用不同的ckpt进行评价,结果却是一样的
请问可能的原因有哪些,如何解决,期待您的回复,谢谢!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the YAML configuration and evaluation command shown in the issue, then trace how Helm passes the checkpoint into the Docker evaluation run. Compare the configuration and logs for two checkpoints; done means the selected fine-tuned checkpoint is loaded and produces distinguishable evaluation results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, helm, python
- Domain
- devops, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100