alibaba / alibaba/FederatedScope

使用helm对微调后的模型进行评估,微调后的参数没有正确加载

Open
#794 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
261
PR merge metrics
No merged PRs in 30d

Description

使用的模型是opt-2.7b,使用lora进行微调,数据集是dolly15k,在docker容器里进行评估。
![Image](https://github.com/user-attachments/assets/7621c47e-c3e6-4f69-a2d6-101011172933)
使用如图的yaml文件,评价命令如下
![Image](https://github.com/user-attachments/assets/f6c37b86-825f-42b8-93ed-4bceb8e53ead)
ckpt文件没有正确加载到评估里,导致使用不同的ckpt进行评价,结果却是一样的
请问可能的原因有哪些,如何解决,期待您的回复,谢谢!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the YAML configuration and evaluation command shown in the issue, then trace how Helm passes the checkpoint into the Docker evaluation run. Compare the configuration and logs for two checkpoints; done means the selected fine-tuned checkpoint is loaded and produces distinguishable evaluation results.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, helm, python
Domain
devops, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.