open-compass / open-compass/opencompass
[Feature] Some suggestion about design
Open
@Leymore is already working on this.
Since Dec 20, 2023.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
描述该功能
这一段的逻辑是不是可以放在 https://github.com/open-compass/opencompass/blob/97c2068bd9b21ac2b30177db6531554f4695bc51/opencompass/models/base.py#L132 里？
_extract_role_pred 看上去将Chat模型的回答中提取出 begin_token 与 end_token 中间的部分，放在模型中似乎更合理。
'pred_role' 看上去只是指示使用 meta_tmplate中的哪一个角色的begin_token\end_token，本质上还是使用最后一段话，我认为不如直接约定为 'BOT' 或者 ‘assistant’的begin_token\end_token。
考虑到的点：
- openicl_eval 里的这段逻辑有些奇怪，放在model里面合理很多。
- predction中不会存在特殊的toekn
- chatinferencer 使用时不用在infer中间去除这些特殊的token
是否希望自己实现该功能？
- 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.