open-compass / open-compass/opencompass

[Bug] configs/datasets中path有的写的是相对路径,有的是Datasets_ids,会导致后续数据集加载错误,按目前版本都应该是Datasets_ids

Open
#1,471 3 comments 1 reaction 1 assignee View on GitHub

@acylam is already working on this.

Since Aug 29, 2024.

Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

先决条件
问题类型

我修改了代码(配置不视为代码),或者我正在处理我自己的任务/模型/数据集。

环境

cuda=12.1、torch=2.4、python3.10

重现问题 - 代码/配置示例
    OpenFinData_datasets.append(
        dict(
            type=OpenFinDataDataset,
            path='./data/openfindata_release',
            name=_name,
            abbr='OpenFinData-' + _name,
            reader_cfg=dict(
                input_columns=['question', 'A', 'B', 'C'],
                output_column='answer'),
            infer_cfg=OpenFinData_infer_cfg,
            eval_cfg=OpenFinData_eval_cfg,
        ))

ARC_c_datasets = [
dict(
abbr='ARC-c',
type=ARCDataset,
path='opencompass/ai2_arc-dev',
name='ARC-Challenge',
reader_cfg=ARC_c_reader_cfg,
infer_cfg=ARC_c_infer_cfg,
eval_cfg=ARC_c_eval_cfg,
)
]

重现问题 - 命令或脚本

python run.py --datasets --models

重现问题 - 错误信息
    OpenFinData_datasets.append(
        dict(
            type=OpenFinDataDataset,
            path='./data/openfindata_release',
            name=_name,
            abbr='OpenFinData-' + _name,
            reader_cfg=dict(
                input_columns=['question', 'A', 'B', 'C'],
                output_column='answer'),
            infer_cfg=OpenFinData_infer_cfg,
            eval_cfg=OpenFinData_eval_cfg,
        ))

ARC_c_datasets = [
dict(
abbr='ARC-c',
type=ARCDataset,
path='opencompass/ai2_arc-dev',
name='ARC-Challenge',
reader_cfg=ARC_c_reader_cfg,
infer_cfg=ARC_c_infer_cfg,
eval_cfg=ARC_c_eval_cfg,
)
]

其他信息

本地加载的方式,path应该都传Dataset_ids,通过Dataset_ids找到地址,而不是相对路径

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.