open-compass / open-compass/opencompass
[Bug] Ceval_gen not compare for modelscope
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
common
Reproduces the problem - code/configuration sample
python run.py --datasets ceval_gen_2daf24 --hf-path /media/fangzhen/DATA/zth/2025-04-train-LLM/LLaMA-Factory-main/saves/Qwen-1.5B/pt --debug --hf-type chat
Reproduces the problem - command or script
python run.py --datasets ceval_gen_2daf24 --hf-path /media/fangzhen/DATA/zth/2025-04-train-LLM/LLaMA-Factory-main/saves/Qwen-1.5B/pt --debug --hf-type chat
Reproduces the problem - error message
04/04 16:30:38 - OpenCompass - INFO - Loading ceval_gen_2daf24: /media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/configs/./datasets/ceval/ceval_gen_2daf24.py
04/04 16:30:38 - OpenCompass - INFO - Loading example: /media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/configs/./summarizers/example.py
04/04 16:30:38 - OpenCompass - INFO - Current exp folder: outputs/default/20250404_163038
04/04 16:30:38 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
Traceback (most recent call last):
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/utils/import_utils.py", line 462, in _get_module
return importlib.import_module('.' + module_name, self.name)
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/msdatasets/init.py", line 2, in
from modelscope.msdatasets.ms_dataset import MsDataset
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/msdatasets/ms_dataset.py", line 25, in
from modelscope.msdatasets.utils.hf_datasets_util import load_dataset_with_ctx
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 19, in
from datasets.data_files import (
ImportError: cannot import name 'get_metadata_patterns' from 'datasets.data_files' (/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/datasets/data_files.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/run.py", line 4, in
main()
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/cli/main.py", line 328, in main
tasks = partitioner(cfg)
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/partitioners/base.py", line 83, in call
tasks = self.partition(**model_and_dataset_args,
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/partitioners/num_worker.py", line 69, in partition
dataset_size = self.get_size(dataset)
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/partitioners/num_worker.py", line 141, in get_size
dataset = build_dataset_from_cfg(dataset)
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/utils/build.py", line 12, in build_dataset_from_cfg
return LOAD_DATASET.build(dataset_cfg)
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/datasets/base.py", line 17, in init
dataset = self.load(**kwargs)
File "/media/fangzhen/DATA/zth/2025-04-train-LLM/opencompass/opencompass/datasets/ceval.py", line 22, in load
from modelscope import MsDataset
File "", line 1075, in _handle_fromlist
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/utils/import_utils.py", line 437, in getattr
module = self._get_module(self._class_to_module[name])
File "/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/modelscope/utils/import_utils.py", line 464, in _get_module
raise RuntimeError(
RuntimeError: Failed to import modelscope.msdatasets because of the following error (look up to see its traceback):
cannot import name 'get_metadata_patterns' from 'datasets.data_files' (/home/fangzhen/anaconda3/envs/opencompass/lib/python3.10/site-packages/datasets/data_files.py)
Other information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.