open-compass / open-compass/opencompass

请问我想对cmmlu这类选择题的模板进行更改,但是system变量无法赋值

Open
#457 2 comments 0 reactions 1 assignee View on GitHub

@tonysy is already working on this.

Since Oct 8, 2023.

Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

Prerequisite
Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

cmmlu_infer_cfg = dict(

     ice_template=dict(
          type=PromptTemplate,
          template=dict(
              begin=[dict(role='SYSTEM', fallback_role='HUMAN', prompt='以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项。'),'</E>'],
             #begin="</E>",
              round=[
                  dict(
                      role="HUMAN",
                      prompt=
                      f"USER: {{question}}\nA. {{A}}\nB. {{B}}\nC. {{C}}\nD. {{D}}"
                  ),
                  dict(role="BOT", prompt=' ASSISTANT: 答案是: {answer}</s>'),
              ]),
          ice_token="</E>",
     ),

)

Reproduces the problem - code/configuration sample

请问怎么让begin里的_ch_name进行赋值呢,我看只有round里面会赋值。或者我应该怎么写比较好呢。”以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项“在fewshot中只在最前面出现一次

Reproduces the problem - command or script

1

Reproduces the problem - error message

1

Other information

1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.