open-compass / open-compass/opencompass

[Bug] TurboMindModelwithChatTemplate中tm_model变量可能未初始化导致报错

Open
#1,384 1 comment 0 reactions 1 assignee View on GitHub

@MaiziXiao is already working on this.

Since Jul 31, 2024.

Bug:P1
Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

Prerequisite
Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

python tools/prompt_viewer.py gsm8k_llama3_8b_ins.py

Reproduces the problem - code/configuration sample

gsm8k_llama3_8b_ins.py

from mmengine.config import read_base
from opencompass.partitioners import NaivePartitioner, SizePartitioner
from opencompass.tasks import OpenICLInferTask, OpenICLEvalTask
from opencompass.runners import LocalRunner


with read_base():
    # 数学
    from ..datasets.gsm8k.gsm8k_gen import gsm8k_datasets as data
    from .model_select import models

datasets = [*data]

infer = dict(
    partitioner=dict(type=SizePartitioner, max_task_size=5000),
    runner=dict(
        type=LocalRunner,
        max_num_workers=4,
        task=dict(type=OpenICLInferTask)),
)

eval = dict(
    partitioner=dict(type=NaivePartitioner),
    runner=dict(
        type=LocalRunner,
        task=dict(type=OpenICLEvalTask),
        max_num_workers=16,
    )
)
from opencompass.models import TurboMindModelwithChatTemplate

models = [
    dict(
        type=TurboMindModelwithChatTemplate,
        abbr='llama-3-8b-instruct-turbomind',
        path='meta-llama/Meta-Llama-3-8B-Instruct',
        engine_config=dict(max_batch_size=16, tp=1),
        gen_config=dict(top_k=1, temperature=1e-6, top_p=0.9, max_new_tokens=1024),
        max_seq_len=7168,
        max_out_len=1024,
        batch_size=16,
        run_cfg=dict(num_gpus=1),
        stop_words=['<|end_of_text|>', '<|eot_id|>'],
    )
]
Reproduces the problem - command or script

在主目录执行命令:
python tools/prompt_viewer.py gsm8k_llama3_8b_ins.py
就会报错

Reproduces the problem - error message
IMG_6107
Other information

错误原因:turbomind_with_tf_above_v4_33.py文件第54到61行。如果tokenizer_only为True。导致tm_model无法被初始化,但是self.generators = [tm_model.create_instance() for i in range(concurrency)]调用了tm_model,导致报错。刚好prompt_viewer.py满足该条件。

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.