[Bug] v0.3.5版本评测Qwen/Qwen2.5-72B得分显著下降

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• The bug has not been fixed in the latest version.

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

torch==2.2.0+vllm==0.4.0+OpenCompass==0.3.5

Reproduces the problem - code/configuration sample

使用如下评测配置评测Qwen/Qwen2.5-72B'

from mmengine.config import read_base

with read_base():
    from .datasets.collections.leaderboard.qwen import datasets
    from .summarizers.leaderboard import summarizer


from opencompass.models import VLLM, HuggingFaceBaseModel


models = [
    dict(
        type=VLLM,
        abbr='qwen2.5-72b-vllm',
        path='Qwen/Qwen2.5-72B',
        model_kwargs=dict(
            tensor_parallel_size=4,
            gpu_memory_utilization=0.8,  # set this to avoid OOM temporarily
            enforce_eager=True,
        ),
        stop_words=['<|endoftext|>', '<|im_end|>'],
        max_out_len=128,
        max_seq_len=8192,
        batch_size=16,
        generation_kwargs=dict(  # args for vllm.SamplingParams
            temperature=0,  #
        ),
        run_cfg=dict(num_gpus=4),
    )
]

Reproduces the problem - command or script

直接使用run.py运行上面的评测配置文件，部分任务在最新的v0.3.5版本得分较低，相较早先v0.2.5(commit e0d7808)版本得分大幅下降

Reproduces the problem - error message

左为v0.3.5版本得分 vs. 右为早先代码版本得分

Other information

No response