open-compass / open-compass/opencompass
[Bug] no error during the evaluation runtime,but the result was empty, and it said "Floating point exception (core dumped)" in output
@MaiziXiao is already working on this.
Since Mar 4, 2025.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
{'CUDA available': False,
'GCC': 'x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',
'MMEngine': '0.10.6',
'MUSA available': False,
'OpenCV': '4.11.0',
'PyTorch': '2.4.0+cpu',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 9.3\n'
' - C++ Version: 201703\n'
' - Intel(R) oneAPI Math Kernel Library Version '
'2022.2-Product Build 20220804 for Intel(R) 64 '
'architecture applications\n'
' - Intel(R) MKL-DNN v3.4.2 (Git Hash '
'1137e04ec0b5251ca2b4400a4fd3c667ce843d67)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - LAPACK is enabled (usually provided by '
'MKL)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX512\n'
' - Build settings: BLAS_INFO=mkl, '
'BUILD_TYPE=Release, '
'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
'-fabi-version=11 -fvisibility-inlines-hidden '
'-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
'-DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER '
'-DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK '
'-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
'-O2 -fPIC -Wall -Wextra -Werror=return-type '
'-Werror=non-virtual-dtor -Werror=bool-operation '
'-Wnarrowing -Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wno-unused-parameter '
'-Wno-unused-function -Wno-unused-result '
'-Wno-strict-overflow -Wno-strict-aliasing '
'-Wno-stringop-overflow -Wsuggest-override '
'-Wno-psabi -Wno-error=pedantic '
'-Wno-error=old-style-cast -Wno-missing-braces '
'-fdiagnostics-color=always -faligned-new '
'-Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Wno-stringop-overflow, LAPACK_INFO=mkl, '
'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
'PERF_WITH_AVX512=1, TORCH_VERSION=2.4.0, '
'USE_CUDA=0, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, '
'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
'USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, '
'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, '
'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, '
'USE_ROCM_KERNEL_ASSERT=OFF, \n',
'Python': '3.10.12 (main, Jan 17 2025, 14:35:34) [GCC 11.4.0]',
'TorchVision': '0.19.0+cpu',
'lmdeploy': "not installed:No module named 'lmdeploy'",
'numpy_random_seed': 2147483648,
'opencompass': '0.4.0+68a9838',
'sys.platform': 'linux',
'transformers': '4.48.3'}
Reproduces the problem - code/configuration sample
datasets=[
dict(abbr='mmlu_pro_math',
category='math',
eval_cfg=dict(
evaluator=dict(
type='opencompass.openicl.icl_evaluator.AccEvaluator'),
pred_postprocessor=dict(
options='ABCDEFGHIJKLMNOP',
type='opencompass.utils.text_postprocessors.first_option_postprocess')),
infer_cfg=dict(
ice_template=dict(
template=dict(
round=[
dict(prompt='Question:\n{question}\nOptions:\n{options_str}',
role='HUMAN'),
dict(prompt="Answer: Let's think step by step. {cot_content}",
role='BOT'),
]),
type='opencompass.openicl.icl_prompt_template.PromptTemplate'),
inferencer=dict(
type='opencompass.openicl.icl_inferencer.GenInferencer'),
prompt_template=dict(
ice_token='</E>',
template=dict(
begin='</E>',
round=[
dict(prompt='Question:\n{question}\nOptions:\n{options_str}',
role='HUMAN'),
]),
type='opencompass.openicl.icl_prompt_template.PromptTemplate'),
retriever=dict(
fix_id_list=[
0,
1,
2,
3,
4,
],
type='opencompass.openicl.icl_retriever.FixKRetriever')),
path='opencompass/mmlu_pro',
reader_cfg=dict(
input_columns=[
'question',
'cot_content',
'options_str',
],
output_column='answer',
test_split='test',
train_split='validation'),
type='opencompass.datasets.MMLUProDataset'),
hf_deepseek_r1_qwen_7b_model=[
dict(abbr='deepseek-v2-hf',
batch_size=4,
engine_config=dict(
max_batch_size=16,
session_len=16384,
tp=2),
max_out_len=1024,
model_kwargs=dict(
attn_implementation='eager',
device_map='auto',
max_memory=dict(
{0: '75GB',
1: '75GB',
2: '75GB',
3: '75GB',
4: '75GB',
5: '75GB',
6: '75GB',
7: '75GB'}),
torch_dtype='torch.bfloat16'),
path='/data/DeepSeek-R1-Distill-Qwen-7B',
run_cfg=dict(
num_gpus=2),
type='opencompass.models.HuggingFaceBaseModel'),
]
Reproduces the problem - command or script
python run.py examples/eval_mmlu_pro_local.py --max-num-workers 2 --hf-num-gpus 2
Reproduces the problem - error message
02/19 19:34:49 - OpenCompass - INFO - Task [deepseek-v2-hf/mmlu_pro_math_1,deepseek-v2-hf/mmlu_pro_physics_1,deepseek-v2-hf/mmlu_pro_chemistry_1,deepseek-v2-hf/mmlu_pro_law_1,deepseek-v2-hf/mmlu_pro_engineering_1,deepseek-v2-hf/mmlu_pro_other_1,deepseek-v2-hf/mmlu_pro_economics_1,deepseek-v2-hf/mmlu_pro_health_1,deepseek-v2-hf/mmlu_pro_psychology_1,deepseek-v2-hf/mmlu_pro_business_1,deepseek-v2-hf/mmlu_pro_biology_1,deepseek-v2-hf/mmlu_pro_philosophy_1,deepseek-v2-hf/mmlu_pro_computer_science_1,deepseek-v2-hf/mmlu_pro_history_1]
^MLoading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]^MLoading checkpoint shards: 50%|█████ | 1/2 [00:03<00:03, 3.09s/it]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00, 1.56s/it]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00, 1.79s/it]
02/19 19:34:54 - OpenCompass - WARNING - Unused argument engine_config={'max_batch_size': 16, 'session_len': 16384, 'tp': 2}
02/19 19:34:55 - OpenCompass - INFO - Try to load the data from /root/.cache/opencompass/./data/mmlu_pro
02/19 19:34:55 - OpenCompass - INFO - Start inferencing [deepseek-v2-hf/mmlu_pro_math_1]
^M 0%| | 0/675 [00:00<?, ?it/s]^M100%|██████████| 675/675 [00:00<00:00, 4403040.75it/s]
[2025-02-19 19:34:55,986] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-02-19 19:34:55,987] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
^M 0%| | 0/169 [00:00<?, ?it/s]/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:628: UserWarning: do_sample is set to False. However, temperature is set to 0.6 -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset temperature.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:633: UserWarning: do_sample is set to False. However, top_p is set to 0.95 -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset top_p.
warnings.warn(
^M 1%| | 1/169 [00:33<1:34:47, 33.85s/it]^M 1%| | 2/169 [01:06<1:32:45, 33.33s/it]^M 2%|▏ | 3/169 [01:39<1:31:54, 33.22s/it]^M 2%|▏ | 4/169 [02:12<1:31:09, 33.15s/it]^M 3%|▎ | 5/169 [02:46<1:30:32, 33.13s/it]^M 4%|▎ | 6/169 [03:18<1:29:42, 33.02s/it]^M 4%|▍ | 7/169 [03:51<1:29:03, 32.99s/it]^M 5%|▍ | 8/169 [04:24<1:28:23, 32.94s/it]^M 5%|▌ | 9/169 [04:57<1:28:00, 33.00s/it]^M 6%|▌ | 10/169 [05:31<1:27:44, 33.11s/it]^M 7%|▋ | 11/169 [06:04<1:27:17, 33.15s/it]^M 7%|▋ | 12/169 [06:37<1:26:31, 33.06s/it]^M 8%|▊ | 13/169 [07:10<1:26:08, 33.13s/it]^M 8%|▊ | 14/169 [07:43<1:25:30, 33.10s/it]^M 9%|▉ | 15/169 [08:16<1:25:07, 33.16s/it]^M 9%|▉ | 16/169 [08:50<1:24:38, 33.19s/it]^M 10%|█ | 17/169 [09:23<1:24:11, 33.23s/it]^M 11%|█ | 18/169 [09:56<1:23:30, 33.18s/it]^M 11%|█ | 19/169 [10:29<1:23:06, 33.24s/it]^M 12%|█▏ | 20/169 [11:04<1:23:26, 33.60s/it]^M 12%|█▏ | 21/169 [11:37<1:22:50, 33.58s/it]^M 13%|█▎ | 22/169 [12:10<1:21:42, 33.35s/it]^M 14%|█▎ | 23/169 [12:43<1:21:05, 33.33s/it]^M 14%|█▍ | 24/169 [13:17<1:20:28, 33.30s/it]^M 15%|█▍ | 25/169 [13:50<1:19:41, 33.20s/it]^M 15%|█▌ | 26/169 [14:23<1:19:11, 33.23s/it]^M 16%|█▌ | 27/169 [14:56<1:18:26, 33.15s/it]^M 17%|█▋ | 28/169 [15:29<1:17:42, 33.07s/it]^M 17%|█▋ | 29/169 [16:02<1:17:06, 33.04s/it]^M 18%|█▊ | 30/169 [16:35<1:16:30, 33.03s/it]^M 18%|█▊ | 31/169 [17:07<1:15:46, 32.94s/it]^M 19%|█▉ | 32/169 [17:40<1:15:09, 32.92s/it]^M 20%|█▉ | 33/169 [18:13<1:14:37, 32.92s/it]^M 20%|██ | 34/169 [18:46<1:14:14, 33.00s/it]^M 21%|██ | 35/169 [19:20<1:13:46, 33.03s/it]^M 21%|██▏ | 36/169 [19:53<1:13:14, 33.04s/it]^M 22%|██▏ | 37/169 [20:25<1:12:32, 32.97s/it]^M 22%|██▏ | 38/169 [20:59<1:12:10, 33.05s/it]^M 23%|██▎ | 39/169 [21:32<1:11:38, 33.07s/it]^M 24%|██▎ | 40/169 [22:05<1:11:09, 33.09s/it]^M 24%|██▍ | 41/169 [22:38<1:10:44, 33.16s/it]^M 25%|██▍ | 42/169 [23:11<1:10:09, 33.15s/it]^M 25%|██▌ | 43/169 [23:44<1:09:33, 33.12s/it]^M 26%|██▌ | 44/169 [24:18<1:08:58, 33.11s/it]^M 27%|██▋ | 45/169 [24:52<1:09:04, 33.42s/it]^M 27%|██▋ | 46/169 [25:25<1:08:44, 33.53s/it]^M 28%|██▊ | 47/169 [25:59<1:08:00, 33.45s/it]^M 28%|██▊ | 48/169 [26:32<1:07:11, 33.32s/it]^M 29%|██▉ | 49/169 [27:05<1:06:20, 33.17s/it]^M 30%|██▉ | 50/169 [27:38<1:05:56, 33.24s/it]^M 30%|███ | 51/169 [28:11<1:05:14, 33.17s/it]^M 31%|███ | 52/169 [28:44<1:04:46, 33.22s/it]^M 31%|███▏ | 53/169 [29:17<1:04:03, 33.13s/it]^M 32%|███▏ | 54/169 [29:50<1:03:31, 33.14s/it]^M 33%|███▎ | 55/169 [30:23<1:02:47, 33.05s/it]^M 33%|███▎ | 56/169 [30:56<1:02:18, 33.09s/it]^M 34%|███▎ | 57/169 [31:29<1:01:41, 33.05s/it]^M 34%|███▍ | 58/169 [32:02<1:01:05, 33.02s/it]^M 35%|███▍ | 59/169 [32:35<1:00:36, 33.06s/it]^M 36%|███▌ | 60/169 [33:08<59:58, 33.01s/it] ^M 36%|███▌ | 61/169 [33:42<59:40, 33.15s/it]^M 37%|███▋ | 62/169 [34:15<59:01, 33.10s/it]^M 37%|███▋ | 63/169 [34:48<58:29, 33.11s/it]^M 38%|███▊ | 64/169 [35:21<57:58, 33.13s/it]^M 38%|███▊ | 65/169 [35:54<57:27, 33.15s/it]^M 39%|███▉ | 66/169 [36:28<57:01, 33.22s/it]^M 40%|███▉ | 67/169 [37:01<56:36, 33.29s/it]^M 40%|████ | 68/169 [37:35<56:06, 33.33s/it]^M 41%|████ | 69/169 [38:08<55:26, 33.27s/it]^M 41%|████▏ | 70/169 [38:41<54:46, 33.20s/it]^M 42%|████▏ | 71/169 [39:14<54:14, 33.21s/it]^M 43%|████▎ | 72/169 [39:47<53:36, 33.16s/it]^M 43%|████▎ | 73/169 [40:20<52:59, 33.12s/it]^M 44%|████▍ | 74/169 [40:53<52:27, 33.14s/it]^M 44%|████▍ | 75/169 [41:27<51:59, 33.19s/it]^M 45%|████▍ | 76/169 [42:00<51:42, 33.36s/it]^M 46%|████▌ | 77/169 [42:34<51:14, 33.42s/it]^M 46%|████▌ | 78/169 [43:07<50:29, 33.29s/it]^M 47%|████▋ | 79/169 [43:40<49:51, 33.23s/it]^M 47%|████▋ | 80/169 [44:13<49:16, 33.22s/it]^M 48%|████▊ | 81/169 [44:46<48:41, 33.19s/it]^M 49%|████▊ | 82/169 [45:19<48:06, 33.18s/it]^M 49%|████▉ | 83/169 [45:53<47:33, 33.18s/it]^M 50%|████▉ | 84/169 [46:25<46:47, 33.03s/it]^M 50%|█████ | 85/169 [46:58<46:13, 33.02s/it]^M 51%|█████ | 86/169 [47:31<45:39, 33.00s/it]^M 51%|█████▏ | 87/169 [48:04<45:06, 33.00s/it]^M 52%|█████▏ | 88/169 [48:37<44:33, 33.01s/it]^M 53%|█████▎ | 89/169 [49:10<43:54, 32.94s/it]^M 53%|█████▎ | 90/169 [49:43<43:23, 32.96s/it]^M 54%|█████▍ | 91/169 [50:16<42:49, 32.95s/it]^M 54%|█████▍ | 92/169 [50:49<42:29, 33.11s/it]^M 55%|█████▌ | 93/169 [51:23<41:57, 33.13s/it]^M 56%|█████▌ | 94/169 [51:57<41:45, 33.41s/it]^M 56%|█████▌ | 95/169 [52:30<41:02, 33.27s/it]^M 57%|█████▋ | 96/169 [53:03<40:21, 33.18s/it]^M 57%|█████▋ | 97/169 [53:36<39:46, 33.15s/it]^M 58%|█████▊ | 98/169 [54:08<39:04, 33.02s/it]^M 59%|█████▊ | 99/169 [54:42<38:39, 33.13s/it]^M 59%|█████▉ | 100/169 [55:15<38:08, 33.16s/it]^M 60%|█████▉ | 101/169 [55:48<37:38, 33.21s/it]^M 60%|██████ | 102/169 [56:21<37:02, 33.17s/it]^M 61%|██████ | 103/169 [56:54<36:26, 33.13s/it]^M 62%|██████▏ | 104/169 [57:29<36:29, 33.68s/it]^M 62%|██████▏ | 105/169 [58:03<35:47, 33.56s/it]^M 63%|██████▎ | 106/169 [58:36<35:03, 33.38s/it]^M 63%|██████▎ | 107/169 [59:08<34:19, 33.22s/it]^M 64%|██████▍ | 108/169 [59:41<33:41, 33.13s/it]^M 64%|██████▍ | 109/169 [1:00:14<33:05, 33.09s/it]^M 65%|██████▌ | 110/169 [1:00:48<32:34, 33.12s/it]^M 66%|██████▌ | 111/169 [1:01:21<31:58, 33.08s/it]^M 66%|██████▋ | 112/169 [1:01:54<31:28, 33.13s/it]^M 67%|██████▋ | 113/169 [1:02:27<30:52, 33.08s/it]^M 67%|██████▋ | 114/169 [1:03:00<30:20, 33.10s/it]^M 68%|██████▊ | 115/169 [1:03:33<29:44, 33.04s/it]^M 69%|██████▊ | 116/169 [1:04:06<29:10, 33.04s/it]^M 69%|██████▉ | 117/169 [1:04:39<28:37, 33.02s/it]^M 70%|██████▉ | 118/169 [1:05:12<28:03, 33.01s/it]^M 70%|███████ | 119/169 [1:05:45<27:30, 33.00s/it]^M 71%|███████ | 120/169 [1:06:18<27:00, 33.07s/it]^M 72%|███████▏ | 121/169 [1:06:51<26:29, 33.11s/it]^M 72%|███████▏ | 122/169 [1:07:24<25:55, 33.09s/it]^M 73%|███████▎ | 123/169 [1:07:58<25:29, 33.25s/it]^M 73%|███████▎ | 124/169 [1:08:32<25:03, 33.41s/it]^M 74%|███████▍ | 125/169 [1:09:05<24:26, 33.33s/it]^M 75%|███████▍ | 126/169 [1:09:38<23:55, 33.39s/it]^M 75%|███████▌ | 127/169 [1:10:11<23:17, 33.28s/it]^M 76%|███████▌ | 128/169 [1:10:44<22:39, 33.16s/it]^M 76%|███████▋ | 129/169 [1:11:18<22:08, 33.22s/it]^M 77%|███████▋ | 130/169 [1:11:51<21:33, 33.15s/it]^M 78%|███████▊ | 131/169 [1:12:23<20:55, 33.04s/it]^M 78%|███████▊ | 132/169 [1:12:57<20:23, 33.07s/it]^M 79%|███████▊ | 133/169 [1:13:30<19:50, 33.07s/it]^M 79%|███████▉ | 134/169 [1:14:03<19:23, 33.24s/it]^M 80%|███████▉ | 135/169 [1:14:36<18:49, 33.22s/it]^M 80%|████████ | 136/169 [1:15:09<18:13, 33.15s/it]^M 81%|████████ | 137/169 [1:15:43<17:44, 33.27s/it]^M 82%|████████▏ | 138/169 [1:16:16<17:09, 33.20s/it]^M 82%|████████▏ | 139/169 [1:16:49<16:35, 33.17s/it]^M 83%|████████▎ | 140/169 [1:17:22<16:02, 33.18s/it]^M 83%|████████▎ | 141/169 [1:17:55<15:28, 33.16s/it]^M 84%|████████▍ | 142/169 [1:18:28<14:54, 33.13s/it]^M 85%|████████▍ | 143/169 [1:19:02<14:21, 33.15s/it]^M 85%|████████▌ | 144/169 [1:19:35<13:52, 33.30s/it]^M 86%|████████▌ | 145/169 [1:20:08<13:16, 33.20s/it]^M 86%|████████▋ | 146/169 [1:20:41<12:41, 33.11s/it]^M 87%|████████▋ | 147/169 [1:21:14<12:08, 33.13s/it]^M 88%|████████▊ | 148/169 [1:21:48<11:36, 33.15s/it]^M 88%|████████▊ | 149/169 [1:22:21<11:04, 33.20s/it]^M 89%|████████▉ | 150/169 [1:22:54<10:30, 33.20s/it]^M 89%|████████▉ | 151/169 [1:23:27<09:57, 33.22s/it]^M 90%|████████▉ | 152/169 [1:24:00<09:24, 33.19s/it]^M 91%|█████████ | 153/169 [1:24:34<08:54, 33.41s/it]^M 91%|█████████ | 154/169 [1:25:08<08:21, 33.42s/it]^M 92%|█████████▏| 155/169 [1:25:41<07:46, 33.32s/it]^M 92%|█████████▏| 156/169 [1:26:14<07:11, 33.17s/it]^M 93%|█████████▎| 157/169 [1:26:47<06:37, 33.09s/it]^M 93%|█████████▎| 158/169 [1:27:20<06:03, 33.04s/it]^M 94%|█████████▍| 159/169 [1:27:53<05:30, 33.03s/it]^M 95%|█████████
………………
02/20 07:06:53 - OpenCompass - INFO - Try to load the data from /root/.cache/opencompass/./data/mmlu_pro
02/20 07:06:53 - OpenCompass - INFO - Start inferencing [deepseek-v2-hf/mmlu_pro_history_1]
^M 0%| | 0/190 [00:00<?, ?it/s]^M100%|██████████| 190/190 [00:00<00:00, 3187671.04it/s]
[2025-02-20 07:06:53,901] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-02-20 07:06:53,901] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
^M 0%| | 0/48 [00:00<?, ?it/s]^M 2%|▏ | 1/48 [00:30<23:31, 30.04s/it]^M 4%|▍ | 2/48 [00:45<16:34, 21.62s/it]^M 6%|▋ | 3/48 [01:20<20:46, 27.69s/it]^M 8%|▊ | 4/48 [01:55<22:30, 30.69s/it]^M 10%|█ | 5/48 [02:11<17:57, 25.05s/it]^M 12%|█▎ | 6/48 [02:35<17:18, 24.73s/it]^M 15%|█▍ | 7/48 [02:56<16:15, 23.78s/it]^M 17%|█▋ | 8/48 [03:32<18:22, 27.56s/it]^M 19%|█▉ | 9/48 [04:08<19:33, 30.08s/it]^M 21%|██ | 10/48 [04:26<16:47, 26.52s/it]^M 23%|██▎ | 11/48 [05:00<17:47, 28.86s/it]^M 25%|██▌ | 12/48 [05:35<18:25, 30.70s/it]^M 27%|██▋ | 13/48 [06:11<18:48, 32.24s/it]^M 29%|██▉ | 14/48 [06:46<18:44, 33.06s/it]^M 31%|███▏ | 15/48 [07:21<18:31, 33.67s/it]^M 33%|███▎ | 16/48 [07:36<14:52, 27.89s/it]^M 35%|███▌ | 17/48 [07:51<12:30, 24.21s/it]^M 38%|███▊ | 18/48 [08:05<10:34, 21.16s/it]^M 40%|███▉ | 19/48 [08:40<12:14, 25.33s/it]^M 42%|████▏ | 20/48 [09:15<13:10, 28.21s/it]^M 44%|████▍ | 21/48 [09:51<13:45, 30.58s/it]^M 46%|████▌ | 22/48 [10:26<13:49, 31.90s/it]^M 48%|████▊ | 23/48 [11:02<13:43, 32.94s/it]^M 50%|█████ | 24/48 [11:36<13:23, 33.47s/it]^M 52%|█████▏ | 25/48 [12:03<12:00, 31.32s/it]^M 54%|█████▍ | 26/48 [12:15<09:26, 25.74s/it]^M 56%|█████▋ | 27/48 [12:51<09:59, 28.53s/it]^M 58%|█████▊ | 28/48 [13:25<10:08, 30.41s/it]^M 60%|██████ | 29/48 [14:01<10:05, 31.86s/it]^M 62%|██████▎ | 30/48 [14:16<08:06, 27.04s/it]^M 65%|██████▍ | 31/48 [14:52<08:25, 29.74s/it]^M 67%|██████▋ | 32/48 [15:02<06:19, 23.73s/it]^M 69%|██████▉ | 33/48 [15:24<05:47, 23.16s/it]^M 71%|███████ | 34/48 [15:58<06:09, 26.38s/it]^M 73%|███████▎ | 35/48 [16:32<06:13, 28.70s/it]^M 75%|███████▌ | 36/48 [17:06<06:02, 30.25s/it]^M 77%|███████▋ | 37/48 [17:22<04:46, 26.06s/it]^M 79%|███████▉ | 38/48 [17:57<04:47, 28.73s/it]^M 81%|████████▏ | 39/48 [18:15<03:48, 25.38s/it]^M 83%|████████▎ | 40/48 [18:36<03:13, 24.13s/it]^M 85%|████████▌ | 41/48 [18:57<02:41, 23.09s/it]^M 88%|████████▊ | 42/48 [19:29<02:36, 26.04s/it]^M 90%|████████▉ | 43/48 [20:04<02:23, 28.69s/it]^M 92%|█████████▏| 44/48 [20:38<02:01, 30.32s/it]^M 94%|█████████▍| 45/48 [21:13<01:35, 31.70s/it]^M 96%|█████████▌| 46/48 [21:50<01:06, 33.05s/it]^M 98%|█████████▊| 47/48 [22:05<00:27, 27.72s/it]^M100%|██████████| 48/48 [22:23<00:00, 24.97s/it]^M100%|██████████| 48/48 [22:23<00:00, 28.00s/it]
02/20 07:29:17 - OpenCompass - INFO - time elapsed: 42868.69s
Other information
there was no error message in both mmlu_pro_math_0.out and mmlu_pro_math_1.out in logs/infer/,but in logs/eval
where there were the same output "Floating point exception (core dumped)" in each file.
and the summary file has:
dataset,version,metric,mode,deepseek-v2-hf
mmlu_pro,-,-,-,-
mmlu_pro_biology,-,-,-,-
mmlu_pro_business,-,-,-,-
mmlu_pro_chemistry,-,-,-,-
mmlu_pro_computer_science,-,-,-,-
mmlu_pro_economics,-,-,-,-
mmlu_pro_engineering,-,-,-,-
mmlu_pro_health,-,-,-,-
mmlu_pro_history,-,-,-,-
mmlu_pro_law,-,-,-,-
mmlu_pro_math,-,-,-,-
mmlu_pro_philosophy,-,-,-,-
mmlu_pro_physics,-,-,-,-
mmlu_pro_psychology,-,-,-,-
mmlu_pro_other,-,-,-,-
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.