open-compass / open-compass/opencompass
[Feature] ModuleNotFoundError: No module named 'opencompass.datasets.lawbench.evaluation_functions'
@Leymore is already working on this.
Since Dec 25, 2023.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
Describe the feature
我执行命令python run.py --datasets ceval_ppl mmlu_ppl --hf-path /T106/LLM_model/llama-7b --model-kwargs device_map='auto' --tokenizer-kwargs padding_side='left' truncation='left' use_fast=False --max-out-len 100 --max-seq-len 2048 --batch-size 8 --no-batch-padding --num-gpus 1
...................................
98%|████████████████████████████████████████████████████████████████████████████████████████████▎ | 107/109 [04:13<00:01, 1.99it/s]12/25 12:59:23 - OpenCompass - WARNING - task OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics] fail, see
./outputs/default/20231225_125408/logs/eval/opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics.out
12/25 12:59:23 - OpenCompass - WARNING - task OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_us_foreign_policy] fail, see
./outputs/default/20231225_125408/logs/eval/opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_us_foreign_policy.out
100%|██████████████████████████████████████████████████████████████████████████████████████████████| 109/109 [04:13<00:00, 2.33s/it]
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/ceval-computer_network] failed with code 1
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/ceval-operating_system] failed with code 1
...................
...................
...................
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics] failed with code 1
dataset
...............................
File "", line 241, in _call_with_frames_removed
File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/init.py", line 51, in
from .lawbench import * # noqa: F401, F403
File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/lawbench/init.py", line 1, in
from .lawbench import LawBenchDataset # noqa: F401
File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/lawbench/lawbench.py", line 10, in
from .evaluation_functions import (cjft, flzx, ftcs, jdzy, jec_ac, jec_kd,
ModuleNotFoundError: No module named 'opencompass.datasets.lawbench.evaluation_functions'
Will you implement it?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.