open-compass / open-compass/opencompass

[Bug] Unable to connect to Hugging Face's MultiPLE-Evaluator

Open
#2,229 0 comments 0 reactions 2 assignees View on GitHub

@Zhudongsheng75 is already working on this.

Since Aug 14, 2025.

Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

Prerequisite
Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': True,
'CUDA_HOME': '/usr/local/cuda-12.4',
'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',
'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H100 80GB HBM3',
'MMEngine': '0.10.7',
'MUSA available': False,
'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131',
'OpenCV': '4.11.0',
'PyTorch': '2.5.0',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 11.4\n'
' - C++ Version: 201703\n'
' - Intel(R) MKL-DNN v3.5.3 (Git Hash '
'66f0cb9eb66affd2da3bf5f8d897376f04aae6af)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX512\n'
' - CUDA Runtime 12.4\n'
' - NVCC architecture flags: '
'-gencode;arch=compute_90,code=sm_90\n'
' - CuDNN 90.1\n'
' - Build settings: BUILD_TYPE=Release, '
'CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, '
'CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= '
'-D_GLIBCXX_USE_CXX11_ABI=1 '
'-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
'-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER '
'-DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM '
'-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK '
'-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC '
'-Wall -Wextra -Werror=return-type '
'-Werror=non-virtual-dtor '
'-Werror=range-loop-construct '
'-Werror=bool-operation -Wnarrowing '
'-Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wno-unused-parameter '
'-Wno-strict-overflow -Wno-strict-aliasing '
'-Wno-stringop-overflow -Wsuggest-override '
'-Wno-psabi -Wno-error=old-style-cast '
'-Wno-missing-braces -fdiagnostics-color=always '
'-faligned-new -Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Wno-stringop-overflow, PERF_WITH_AVX=1, '
'PERF_WITH_AVX2=1, TORCH_VERSION=2.5.0, '
'USE_CUDA=1, USE_CUDNN=ON, USE_CUSPARSELT=OFF, '
'USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, '
'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, '
'USE_MKL=OFF, USE_MKLDNN=ON, USE_MPI=OFF, '
'USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, '
'USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, \n',
'Python': '3.10.18 (main, Jun 5 2025, 13:14:17) [GCC 11.2.0]',
'lmdeploy': "not installed:No module named 'lmdeploy'",
'numpy_random_seed': 2147483648,
'opencompass': '0.4.2+',
'sys.platform': 'linux',
'transformers': '4.51.3'}

Reproduces the problem - code/configuration sample

from gradio_client import Client

ip_address='https://opencompass-multiple-evaluator.hf.space'
client = Client(ip_address)

Reproduces the problem - command or script

None

Reproduces the problem - error message

Loaded as API: https://opencompass-multiple-evaluator.hf.space/
Traceback (most recent call last):
File "/home/workspace/test.py", line 4, in
client = Client(ip_address)
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 153, in init
self.config = self._get_config()
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 890, in _get_config
return r.json()
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/httpx/_models.py", line 766, in json
return jsonlib.loads(self.content, **kwargs)
File "/root/miniconda/envs/torch/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Other information

The MultiPLE-Evaluator Space on Hugging Face is stuck at "Preparing Space" and cannot be accessed

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.