open-compass / open-compass/opencompass

[Bug] no error during the evaluation runtime,but the result was empty, and it said "Floating point exception (core dumped)" in output

Open
#1,903 3 comments 0 reactions 1 assignee View on GitHub

@MaiziXiao is already working on this.

Since Mar 4, 2025.

Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

Prerequisite
Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': False,
'GCC': 'x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',
'MMEngine': '0.10.6',
'MUSA available': False,
'OpenCV': '4.11.0',
'PyTorch': '2.4.0+cpu',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 9.3\n'
' - C++ Version: 201703\n'
' - Intel(R) oneAPI Math Kernel Library Version '
'2022.2-Product Build 20220804 for Intel(R) 64 '
'architecture applications\n'
' - Intel(R) MKL-DNN v3.4.2 (Git Hash '
'1137e04ec0b5251ca2b4400a4fd3c667ce843d67)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - LAPACK is enabled (usually provided by '
'MKL)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX512\n'
' - Build settings: BLAS_INFO=mkl, '
'BUILD_TYPE=Release, '
'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
'-fabi-version=11 -fvisibility-inlines-hidden '
'-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
'-DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER '
'-DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK '
'-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
'-O2 -fPIC -Wall -Wextra -Werror=return-type '
'-Werror=non-virtual-dtor -Werror=bool-operation '
'-Wnarrowing -Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wno-unused-parameter '
'-Wno-unused-function -Wno-unused-result '
'-Wno-strict-overflow -Wno-strict-aliasing '
'-Wno-stringop-overflow -Wsuggest-override '
'-Wno-psabi -Wno-error=pedantic '
'-Wno-error=old-style-cast -Wno-missing-braces '
'-fdiagnostics-color=always -faligned-new '
'-Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Wno-stringop-overflow, LAPACK_INFO=mkl, '
'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
'PERF_WITH_AVX512=1, TORCH_VERSION=2.4.0, '
'USE_CUDA=0, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, '
'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
'USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, '
'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, '
'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, '
'USE_ROCM_KERNEL_ASSERT=OFF, \n',
'Python': '3.10.12 (main, Jan 17 2025, 14:35:34) [GCC 11.4.0]',
'TorchVision': '0.19.0+cpu',
'lmdeploy': "not installed:No module named 'lmdeploy'",
'numpy_random_seed': 2147483648,
'opencompass': '0.4.0+68a9838',
'sys.platform': 'linux',
'transformers': '4.48.3'}

Reproduces the problem - code/configuration sample
datasets=[
    dict(abbr='mmlu_pro_math',
        category='math',
        eval_cfg=dict(
            evaluator=dict(
                type='opencompass.openicl.icl_evaluator.AccEvaluator'),
            pred_postprocessor=dict(
                options='ABCDEFGHIJKLMNOP',
                type='opencompass.utils.text_postprocessors.first_option_postprocess')),
        infer_cfg=dict(
            ice_template=dict(
                template=dict(
                    round=[
                        dict(prompt='Question:\n{question}\nOptions:\n{options_str}',
                            role='HUMAN'),
                        dict(prompt="Answer: Let's think step by step. {cot_content}",
                            role='BOT'),
                        ]),
                type='opencompass.openicl.icl_prompt_template.PromptTemplate'),
            inferencer=dict(
                type='opencompass.openicl.icl_inferencer.GenInferencer'),
            prompt_template=dict(
                ice_token='</E>',
                template=dict(
                    begin='</E>',
                    round=[
                        dict(prompt='Question:\n{question}\nOptions:\n{options_str}',
                            role='HUMAN'),
                        ]),
                type='opencompass.openicl.icl_prompt_template.PromptTemplate'),
            retriever=dict(
                fix_id_list=[
                    0,
                    1,
                    2,
                    3,
                    4,
                    ],
                type='opencompass.openicl.icl_retriever.FixKRetriever')),
        path='opencompass/mmlu_pro',
        reader_cfg=dict(
            input_columns=[
                'question',
                'cot_content',
                'options_str',
                ],
            output_column='answer',
            test_split='test',
            train_split='validation'),
        type='opencompass.datasets.MMLUProDataset'),
hf_deepseek_r1_qwen_7b_model=[
    dict(abbr='deepseek-v2-hf',
        batch_size=4,
        engine_config=dict(
            max_batch_size=16,
            session_len=16384,
            tp=2),
        max_out_len=1024,
        model_kwargs=dict(
            attn_implementation='eager',
            device_map='auto',
            max_memory=dict(
                {0: '75GB',
                1: '75GB',
                2: '75GB',
                3: '75GB',
                4: '75GB',
                5: '75GB',
                6: '75GB',
                7: '75GB'}),
            torch_dtype='torch.bfloat16'),
        path='/data/DeepSeek-R1-Distill-Qwen-7B',
        run_cfg=dict(
            num_gpus=2),
        type='opencompass.models.HuggingFaceBaseModel'),
    ]
Reproduces the problem - command or script

python run.py examples/eval_mmlu_pro_local.py --max-num-workers 2 --hf-num-gpus 2

Reproduces the problem - error message

02/19 19:34:49 - OpenCompass - INFO - Task [deepseek-v2-hf/mmlu_pro_math_1,deepseek-v2-hf/mmlu_pro_physics_1,deepseek-v2-hf/mmlu_pro_chemistry_1,deepseek-v2-hf/mmlu_pro_law_1,deepseek-v2-hf/mmlu_pro_engineering_1,deepseek-v2-hf/mmlu_pro_other_1,deepseek-v2-hf/mmlu_pro_economics_1,deepseek-v2-hf/mmlu_pro_health_1,deepseek-v2-hf/mmlu_pro_psychology_1,deepseek-v2-hf/mmlu_pro_business_1,deepseek-v2-hf/mmlu_pro_biology_1,deepseek-v2-hf/mmlu_pro_philosophy_1,deepseek-v2-hf/mmlu_pro_computer_science_1,deepseek-v2-hf/mmlu_pro_history_1]
^MLoading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]^MLoading checkpoint shards: 50%|█████ | 1/2 [00:03<00:03, 3.09s/it]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00, 1.56s/it]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00, 1.79s/it]
02/19 19:34:54 - OpenCompass - WARNING - Unused argument engine_config={'max_batch_size': 16, 'session_len': 16384, 'tp': 2}
02/19 19:34:55 - OpenCompass - INFO - Try to load the data from /root/.cache/opencompass/./data/mmlu_pro
02/19 19:34:55 - OpenCompass - INFO - Start inferencing [deepseek-v2-hf/mmlu_pro_math_1]
^M 0%| | 0/675 [00:00<?, ?it/s]^M100%|██████████| 675/675 [00:00<00:00, 4403040.75it/s]
[2025-02-19 19:34:55,986] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-02-19 19:34:55,987] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
^M 0%| | 0/169 [00:00<?, ?it/s]/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:628: UserWarning: do_sample is set to False. However, temperature is set to 0.6 -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset temperature.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:633: UserWarning: do_sample is set to False. However, top_p is set to 0.95 -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset top_p.
warnings.warn(
^M 1%| | 1/169 [00:33<1:34:47, 33.85s/it]^M 1%| | 2/169 [01:06<1:32:45, 33.33s/it]^M 2%|▏ | 3/169 [01:39<1:31:54, 33.22s/it]^M 2%|▏ | 4/169 [02:12<1:31:09, 33.15s/it]^M 3%|▎ | 5/169 [02:46<1:30:32, 33.13s/it]^M 4%|▎ | 6/169 [03:18<1:29:42, 33.02s/it]^M 4%|▍ | 7/169 [03:51<1:29:03, 32.99s/it]^M 5%|▍ | 8/169 [04:24<1:28:23, 32.94s/it]^M 5%|▌ | 9/169 [04:57<1:28:00, 33.00s/it]^M 6%|▌ | 10/169 [05:31<1:27:44, 33.11s/it]^M 7%|▋ | 11/169 [06:04<1:27:17, 33.15s/it]^M 7%|▋ | 12/169 [06:37<1:26:31, 33.06s/it]^M 8%|▊ | 13/169 [07:10<1:26:08, 33.13s/it]^M 8%|▊ | 14/169 [07:43<1:25:30, 33.10s/it]^M 9%|▉ | 15/169 [08:16<1:25:07, 33.16s/it]^M 9%|▉ | 16/169 [08:50<1:24:38, 33.19s/it]^M 10%|█ | 17/169 [09:23<1:24:11, 33.23s/it]^M 11%|█ | 18/169 [09:56<1:23:30, 33.18s/it]^M 11%|█ | 19/169 [10:29<1:23:06, 33.24s/it]^M 12%|█▏ | 20/169 [11:04<1:23:26, 33.60s/it]^M 12%|█▏ | 21/169 [11:37<1:22:50, 33.58s/it]^M 13%|█▎ | 22/169 [12:10<1:21:42, 33.35s/it]^M 14%|█▎ | 23/169 [12:43<1:21:05, 33.33s/it]^M 14%|█▍ | 24/169 [13:17<1:20:28, 33.30s/it]^M 15%|█▍ | 25/169 [13:50<1:19:41, 33.20s/it]^M 15%|█▌ | 26/169 [14:23<1:19:11, 33.23s/it]^M 16%|█▌ | 27/169 [14:56<1:18:26, 33.15s/it]^M 17%|█▋ | 28/169 [15:29<1:17:42, 33.07s/it]^M 17%|█▋ | 29/169 [16:02<1:17:06, 33.04s/it]^M 18%|█▊ | 30/169 [16:35<1:16:30, 33.03s/it]^M 18%|█▊ | 31/169 [17:07<1:15:46, 32.94s/it]^M 19%|█▉ | 32/169 [17:40<1:15:09, 32.92s/it]^M 20%|█▉ | 33/169 [18:13<1:14:37, 32.92s/it]^M 20%|██ | 34/169 [18:46<1:14:14, 33.00s/it]^M 21%|██ | 35/169 [19:20<1:13:46, 33.03s/it]^M 21%|██▏ | 36/169 [19:53<1:13:14, 33.04s/it]^M 22%|██▏ | 37/169 [20:25<1:12:32, 32.97s/it]^M 22%|██▏ | 38/169 [20:59<1:12:10, 33.05s/it]^M 23%|██▎ | 39/169 [21:32<1:11:38, 33.07s/it]^M 24%|██▎ | 40/169 [22:05<1:11:09, 33.09s/it]^M 24%|██▍ | 41/169 [22:38<1:10:44, 33.16s/it]^M 25%|██▍ | 42/169 [23:11<1:10:09, 33.15s/it]^M 25%|██▌ | 43/169 [23:44<1:09:33, 33.12s/it]^M 26%|██▌ | 44/169 [24:18<1:08:58, 33.11s/it]^M 27%|██▋ | 45/169 [24:52<1:09:04, 33.42s/it]^M 27%|██▋ | 46/169 [25:25<1:08:44, 33.53s/it]^M 28%|██▊ | 47/169 [25:59<1:08:00, 33.45s/it]^M 28%|██▊ | 48/169 [26:32<1:07:11, 33.32s/it]^M 29%|██▉ | 49/169 [27:05<1:06:20, 33.17s/it]^M 30%|██▉ | 50/169 [27:38<1:05:56, 33.24s/it]^M 30%|███ | 51/169 [28:11<1:05:14, 33.17s/it]^M 31%|███ | 52/169 [28:44<1:04:46, 33.22s/it]^M 31%|███▏ | 53/169 [29:17<1:04:03, 33.13s/it]^M 32%|███▏ | 54/169 [29:50<1:03:31, 33.14s/it]^M 33%|███▎ | 55/169 [30:23<1:02:47, 33.05s/it]^M 33%|███▎ | 56/169 [30:56<1:02:18, 33.09s/it]^M 34%|███▎ | 57/169 [31:29<1:01:41, 33.05s/it]^M 34%|███▍ | 58/169 [32:02<1:01:05, 33.02s/it]^M 35%|███▍ | 59/169 [32:35<1:00:36, 33.06s/it]^M 36%|███▌ | 60/169 [33:08<59:58, 33.01s/it] ^M 36%|███▌ | 61/169 [33:42<59:40, 33.15s/it]^M 37%|███▋ | 62/169 [34:15<59:01, 33.10s/it]^M 37%|███▋ | 63/169 [34:48<58:29, 33.11s/it]^M 38%|███▊ | 64/169 [35:21<57:58, 33.13s/it]^M 38%|███▊ | 65/169 [35:54<57:27, 33.15s/it]^M 39%|███▉ | 66/169 [36:28<57:01, 33.22s/it]^M 40%|███▉ | 67/169 [37:01<56:36, 33.29s/it]^M 40%|████ | 68/169 [37:35<56:06, 33.33s/it]^M 41%|████ | 69/169 [38:08<55:26, 33.27s/it]^M 41%|████▏ | 70/169 [38:41<54:46, 33.20s/it]^M 42%|████▏ | 71/169 [39:14<54:14, 33.21s/it]^M 43%|████▎ | 72/169 [39:47<53:36, 33.16s/it]^M 43%|████▎ | 73/169 [40:20<52:59, 33.12s/it]^M 44%|████▍ | 74/169 [40:53<52:27, 33.14s/it]^M 44%|████▍ | 75/169 [41:27<51:59, 33.19s/it]^M 45%|████▍ | 76/169 [42:00<51:42, 33.36s/it]^M 46%|████▌ | 77/169 [42:34<51:14, 33.42s/it]^M 46%|████▌ | 78/169 [43:07<50:29, 33.29s/it]^M 47%|████▋ | 79/169 [43:40<49:51, 33.23s/it]^M 47%|████▋ | 80/169 [44:13<49:16, 33.22s/it]^M 48%|████▊ | 81/169 [44:46<48:41, 33.19s/it]^M 49%|████▊ | 82/169 [45:19<48:06, 33.18s/it]^M 49%|████▉ | 83/169 [45:53<47:33, 33.18s/it]^M 50%|████▉ | 84/169 [46:25<46:47, 33.03s/it]^M 50%|█████ | 85/169 [46:58<46:13, 33.02s/it]^M 51%|█████ | 86/169 [47:31<45:39, 33.00s/it]^M 51%|█████▏ | 87/169 [48:04<45:06, 33.00s/it]^M 52%|█████▏ | 88/169 [48:37<44:33, 33.01s/it]^M 53%|█████▎ | 89/169 [49:10<43:54, 32.94s/it]^M 53%|█████▎ | 90/169 [49:43<43:23, 32.96s/it]^M 54%|█████▍ | 91/169 [50:16<42:49, 32.95s/it]^M 54%|█████▍ | 92/169 [50:49<42:29, 33.11s/it]^M 55%|█████▌ | 93/169 [51:23<41:57, 33.13s/it]^M 56%|█████▌ | 94/169 [51:57<41:45, 33.41s/it]^M 56%|█████▌ | 95/169 [52:30<41:02, 33.27s/it]^M 57%|█████▋ | 96/169 [53:03<40:21, 33.18s/it]^M 57%|█████▋ | 97/169 [53:36<39:46, 33.15s/it]^M 58%|█████▊ | 98/169 [54:08<39:04, 33.02s/it]^M 59%|█████▊ | 99/169 [54:42<38:39, 33.13s/it]^M 59%|█████▉ | 100/169 [55:15<38:08, 33.16s/it]^M 60%|█████▉ | 101/169 [55:48<37:38, 33.21s/it]^M 60%|██████ | 102/169 [56:21<37:02, 33.17s/it]^M 61%|██████ | 103/169 [56:54<36:26, 33.13s/it]^M 62%|██████▏ | 104/169 [57:29<36:29, 33.68s/it]^M 62%|██████▏ | 105/169 [58:03<35:47, 33.56s/it]^M 63%|██████▎ | 106/169 [58:36<35:03, 33.38s/it]^M 63%|██████▎ | 107/169 [59:08<34:19, 33.22s/it]^M 64%|██████▍ | 108/169 [59:41<33:41, 33.13s/it]^M 64%|██████▍ | 109/169 [1:00:14<33:05, 33.09s/it]^M 65%|██████▌ | 110/169 [1:00:48<32:34, 33.12s/it]^M 66%|██████▌ | 111/169 [1:01:21<31:58, 33.08s/it]^M 66%|██████▋ | 112/169 [1:01:54<31:28, 33.13s/it]^M 67%|██████▋ | 113/169 [1:02:27<30:52, 33.08s/it]^M 67%|██████▋ | 114/169 [1:03:00<30:20, 33.10s/it]^M 68%|██████▊ | 115/169 [1:03:33<29:44, 33.04s/it]^M 69%|██████▊ | 116/169 [1:04:06<29:10, 33.04s/it]^M 69%|██████▉ | 117/169 [1:04:39<28:37, 33.02s/it]^M 70%|██████▉ | 118/169 [1:05:12<28:03, 33.01s/it]^M 70%|███████ | 119/169 [1:05:45<27:30, 33.00s/it]^M 71%|███████ | 120/169 [1:06:18<27:00, 33.07s/it]^M 72%|███████▏ | 121/169 [1:06:51<26:29, 33.11s/it]^M 72%|███████▏ | 122/169 [1:07:24<25:55, 33.09s/it]^M 73%|███████▎ | 123/169 [1:07:58<25:29, 33.25s/it]^M 73%|███████▎ | 124/169 [1:08:32<25:03, 33.41s/it]^M 74%|███████▍ | 125/169 [1:09:05<24:26, 33.33s/it]^M 75%|███████▍ | 126/169 [1:09:38<23:55, 33.39s/it]^M 75%|███████▌ | 127/169 [1:10:11<23:17, 33.28s/it]^M 76%|███████▌ | 128/169 [1:10:44<22:39, 33.16s/it]^M 76%|███████▋ | 129/169 [1:11:18<22:08, 33.22s/it]^M 77%|███████▋ | 130/169 [1:11:51<21:33, 33.15s/it]^M 78%|███████▊ | 131/169 [1:12:23<20:55, 33.04s/it]^M 78%|███████▊ | 132/169 [1:12:57<20:23, 33.07s/it]^M 79%|███████▊ | 133/169 [1:13:30<19:50, 33.07s/it]^M 79%|███████▉ | 134/169 [1:14:03<19:23, 33.24s/it]^M 80%|███████▉ | 135/169 [1:14:36<18:49, 33.22s/it]^M 80%|████████ | 136/169 [1:15:09<18:13, 33.15s/it]^M 81%|████████ | 137/169 [1:15:43<17:44, 33.27s/it]^M 82%|████████▏ | 138/169 [1:16:16<17:09, 33.20s/it]^M 82%|████████▏ | 139/169 [1:16:49<16:35, 33.17s/it]^M 83%|████████▎ | 140/169 [1:17:22<16:02, 33.18s/it]^M 83%|████████▎ | 141/169 [1:17:55<15:28, 33.16s/it]^M 84%|████████▍ | 142/169 [1:18:28<14:54, 33.13s/it]^M 85%|████████▍ | 143/169 [1:19:02<14:21, 33.15s/it]^M 85%|████████▌ | 144/169 [1:19:35<13:52, 33.30s/it]^M 86%|████████▌ | 145/169 [1:20:08<13:16, 33.20s/it]^M 86%|████████▋ | 146/169 [1:20:41<12:41, 33.11s/it]^M 87%|████████▋ | 147/169 [1:21:14<12:08, 33.13s/it]^M 88%|████████▊ | 148/169 [1:21:48<11:36, 33.15s/it]^M 88%|████████▊ | 149/169 [1:22:21<11:04, 33.20s/it]^M 89%|████████▉ | 150/169 [1:22:54<10:30, 33.20s/it]^M 89%|████████▉ | 151/169 [1:23:27<09:57, 33.22s/it]^M 90%|████████▉ | 152/169 [1:24:00<09:24, 33.19s/it]^M 91%|█████████ | 153/169 [1:24:34<08:54, 33.41s/it]^M 91%|█████████ | 154/169 [1:25:08<08:21, 33.42s/it]^M 92%|█████████▏| 155/169 [1:25:41<07:46, 33.32s/it]^M 92%|█████████▏| 156/169 [1:26:14<07:11, 33.17s/it]^M 93%|█████████▎| 157/169 [1:26:47<06:37, 33.09s/it]^M 93%|█████████▎| 158/169 [1:27:20<06:03, 33.04s/it]^M 94%|█████████▍| 159/169 [1:27:53<05:30, 33.03s/it]^M 95%|█████████
………………
02/20 07:06:53 - OpenCompass - INFO - Try to load the data from /root/.cache/opencompass/./data/mmlu_pro
02/20 07:06:53 - OpenCompass - INFO - Start inferencing [deepseek-v2-hf/mmlu_pro_history_1]
^M 0%| | 0/190 [00:00<?, ?it/s]^M100%|██████████| 190/190 [00:00<00:00, 3187671.04it/s]
[2025-02-20 07:06:53,901] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-02-20 07:06:53,901] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
^M 0%| | 0/48 [00:00<?, ?it/s]^M 2%|▏ | 1/48 [00:30<23:31, 30.04s/it]^M 4%|▍ | 2/48 [00:45<16:34, 21.62s/it]^M 6%|▋ | 3/48 [01:20<20:46, 27.69s/it]^M 8%|▊ | 4/48 [01:55<22:30, 30.69s/it]^M 10%|█ | 5/48 [02:11<17:57, 25.05s/it]^M 12%|█▎ | 6/48 [02:35<17:18, 24.73s/it]^M 15%|█▍ | 7/48 [02:56<16:15, 23.78s/it]^M 17%|█▋ | 8/48 [03:32<18:22, 27.56s/it]^M 19%|█▉ | 9/48 [04:08<19:33, 30.08s/it]^M 21%|██ | 10/48 [04:26<16:47, 26.52s/it]^M 23%|██▎ | 11/48 [05:00<17:47, 28.86s/it]^M 25%|██▌ | 12/48 [05:35<18:25, 30.70s/it]^M 27%|██▋ | 13/48 [06:11<18:48, 32.24s/it]^M 29%|██▉ | 14/48 [06:46<18:44, 33.06s/it]^M 31%|███▏ | 15/48 [07:21<18:31, 33.67s/it]^M 33%|███▎ | 16/48 [07:36<14:52, 27.89s/it]^M 35%|███▌ | 17/48 [07:51<12:30, 24.21s/it]^M 38%|███▊ | 18/48 [08:05<10:34, 21.16s/it]^M 40%|███▉ | 19/48 [08:40<12:14, 25.33s/it]^M 42%|████▏ | 20/48 [09:15<13:10, 28.21s/it]^M 44%|████▍ | 21/48 [09:51<13:45, 30.58s/it]^M 46%|████▌ | 22/48 [10:26<13:49, 31.90s/it]^M 48%|████▊ | 23/48 [11:02<13:43, 32.94s/it]^M 50%|█████ | 24/48 [11:36<13:23, 33.47s/it]^M 52%|█████▏ | 25/48 [12:03<12:00, 31.32s/it]^M 54%|█████▍ | 26/48 [12:15<09:26, 25.74s/it]^M 56%|█████▋ | 27/48 [12:51<09:59, 28.53s/it]^M 58%|█████▊ | 28/48 [13:25<10:08, 30.41s/it]^M 60%|██████ | 29/48 [14:01<10:05, 31.86s/it]^M 62%|██████▎ | 30/48 [14:16<08:06, 27.04s/it]^M 65%|██████▍ | 31/48 [14:52<08:25, 29.74s/it]^M 67%|██████▋ | 32/48 [15:02<06:19, 23.73s/it]^M 69%|██████▉ | 33/48 [15:24<05:47, 23.16s/it]^M 71%|███████ | 34/48 [15:58<06:09, 26.38s/it]^M 73%|███████▎ | 35/48 [16:32<06:13, 28.70s/it]^M 75%|███████▌ | 36/48 [17:06<06:02, 30.25s/it]^M 77%|███████▋ | 37/48 [17:22<04:46, 26.06s/it]^M 79%|███████▉ | 38/48 [17:57<04:47, 28.73s/it]^M 81%|████████▏ | 39/48 [18:15<03:48, 25.38s/it]^M 83%|████████▎ | 40/48 [18:36<03:13, 24.13s/it]^M 85%|████████▌ | 41/48 [18:57<02:41, 23.09s/it]^M 88%|████████▊ | 42/48 [19:29<02:36, 26.04s/it]^M 90%|████████▉ | 43/48 [20:04<02:23, 28.69s/it]^M 92%|█████████▏| 44/48 [20:38<02:01, 30.32s/it]^M 94%|█████████▍| 45/48 [21:13<01:35, 31.70s/it]^M 96%|█████████▌| 46/48 [21:50<01:06, 33.05s/it]^M 98%|█████████▊| 47/48 [22:05<00:27, 27.72s/it]^M100%|██████████| 48/48 [22:23<00:00, 24.97s/it]^M100%|██████████| 48/48 [22:23<00:00, 28.00s/it]
02/20 07:29:17 - OpenCompass - INFO - time elapsed: 42868.69s

Other information

there was no error message in both mmlu_pro_math_0.out and mmlu_pro_math_1.out in logs/infer/,but in logs/eval

Image
where there were the same output "Floating point exception (core dumped)" in each file.
and the summary file has:
dataset,version,metric,mode,deepseek-v2-hf
mmlu_pro,-,-,-,-
mmlu_pro_biology,-,-,-,-
mmlu_pro_business,-,-,-,-
mmlu_pro_chemistry,-,-,-,-
mmlu_pro_computer_science,-,-,-,-
mmlu_pro_economics,-,-,-,-
mmlu_pro_engineering,-,-,-,-
mmlu_pro_health,-,-,-,-
mmlu_pro_history,-,-,-,-
mmlu_pro_law,-,-,-,-
mmlu_pro_math,-,-,-,-
mmlu_pro_philosophy,-,-,-,-
mmlu_pro_physics,-,-,-,-
mmlu_pro_psychology,-,-,-,-
mmlu_pro_other,-,-,-,-

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.