open-compass / open-compass/opencompass
请问我想对cmmlu这类选择题的模板进行更改,但是system变量无法赋值
Open
@tonysy is already working on this.
Since Oct 8, 2023.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
cmmlu_infer_cfg = dict(
ice_template=dict(
type=PromptTemplate,
template=dict(
begin=[dict(role='SYSTEM', fallback_role='HUMAN', prompt='以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项。'),'</E>'],
#begin="</E>",
round=[
dict(
role="HUMAN",
prompt=
f"USER: {{question}}\nA. {{A}}\nB. {{B}}\nC. {{C}}\nD. {{D}}"
),
dict(role="BOT", prompt=' ASSISTANT: 答案是: {answer}</s>'),
]),
ice_token="</E>",
),
)
Reproduces the problem - code/configuration sample
请问怎么让begin里的_ch_name进行赋值呢,我看只有round里面会赋值。或者我应该怎么写比较好呢。”以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项“在fewshot中只在最前面出现一次
Reproduces the problem - command or script
1
Reproduces the problem - error message
1
Other information
1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.