open-compass / open-compass/opencompass

[Bug] TypeError: Fields of type "<class 'typing.IO'>" are not supported.

Open
#889 0 comments 0 reactions 1 assignee View on GitHub

@Leymore is already working on this.

Since Feb 19, 2024.

Dominant language
Python
Stars
7.5k
Forks
869
Avg merge
17h 52m
Merged PRs (30d)
13

Description

Prerequisite
Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': True,
'CUDA_HOME': '/usr/local/cuda',
'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0',
'GPU 0,1,2,3,4,5,6,7': 'NVIDIA A800-SXM4-80GB',
'MMEngine': '0.10.3',
'MUSA available': False,
'NVCC': 'Cuda compilation tools, release 11.7, V11.7.64',
'OpenCV': '4.9.0',
'PyTorch': '1.13.1+cu117',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 9.3\n'
' - C++ Version: 201402\n'
' - Intel(R) Math Kernel Library Version '
'2020.0.0 Product Build 20191122 for Intel(R) 64 '
'architecture applications\n'
' - Intel(R) MKL-DNN v2.6.0 (Git Hash '
'52b5f107dd9cf10910aaa19cb47f3abf9b349815)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - LAPACK is enabled (usually provided by '
'MKL)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX2\n'
' - CUDA Runtime 11.7\n'
' - NVCC architecture flags: '
'-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n'
' - CuDNN 8.9.2 (built against CUDA 12.1)\n'
' - Built with CuDNN 8.5\n'
' - Magma 2.6.1\n'
' - Build settings: BLAS_INFO=mkl, '
'BUILD_TYPE=Release, CUDA_VERSION=11.7, '
'CUDNN_VERSION=8.5.0, '
'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
'CXX_FLAGS= -fabi-version=11 -Wno-deprecated '
'-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
'-fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM '
'-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK '
'-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
'-DEDGE_PROFILER_USE_KINETO -O2 -fPIC '
'-Wno-narrowing -Wall -Wextra '
'-Werror=return-type -Werror=non-virtual-dtor '
'-Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wunused-local-typedefs '
'-Wno-unused-parameter -Wno-unused-function '
'-Wno-unused-result -Wno-strict-overflow '
'-Wno-strict-aliasing '
'-Wno-error=deprecated-declarations '
'-Wno-stringop-overflow -Wno-psabi '
'-Wno-error=pedantic -Wno-error=redundant-decls '
'-Wno-error=old-style-cast '
'-fdiagnostics-color=always -faligned-new '
'-Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Werror=cast-function-type '
'-Wno-stringop-overflow, LAPACK_INFO=mkl, '
'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
'PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, '
'USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, '
'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, '
'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, '
'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n',
'Python': '3.10.9 (main, Mar 1 2023, 18:23:06) [GCC 11.2.0]',
'TorchVision': '0.14.1+cu117',
'numpy_random_seed': 2147483648,
'opencompass': '0.2.1+0919b08',
'sys.platform': 'linux'}

Reproduces the problem - code/configuration sample

python run.py --models hf_opt_350m hf_opt_125m --datasets siqa_gen winograd_ppl

Reproduces the problem - command or script

python run.py --models hf_opt_350m hf_opt_125m --datasets siqa_gen winograd_ppl

Reproduces the problem - error message

File "/home/dengkaibiao/opencompass/run.py", line 14, in
from opencompass.utils.run import (exec_mm_infer_runner, fill_eval_cfg,
File "/home/dengkaibiao/opencompass/opencompass/utils/run.py", line 7, in
from opencompass.datasets.custom import make_custom_dataset_config
File "/home/dengkaibiao/opencompass/opencompass/datasets/init.py", line 1, in
from .advglue import * # noqa: F401, F403
File "/home/dengkaibiao/opencompass/opencompass/datasets/advglue.py", line 6, in
from opencompass.openicl.icl_evaluator import AccEvaluator
File "/home/dengkaibiao/opencompass/opencompass/openicl/init.py", line 2, in
from .icl_evaluator import * # noqa
File "/home/dengkaibiao/opencompass/opencompass/openicl/icl_evaluator/init.py", line 1, in
from .icl_agent_evaluator import * # noqa
File "/home/dengkaibiao/opencompass/opencompass/openicl/icl_evaluator/icl_agent_evaluator.py", line 11, in
from opencompass.models import OpenAI
File "/home/dengkaibiao/opencompass/opencompass/models/init.py", line 31, in
from .zhipuai_v2_api import ZhiPuV2AI # noqa: F401
File "/home/dengkaibiao/opencompass/opencompass/models/zhipuai_v2_api.py", line 13, in
from zhipuai.core._errors import APIStatusError, APITimeoutError
File "/home/dengkaibiao/anaconda3/lib/python3.10/site-packages/zhipuai/init.py", line 2, in
from ._client import ZhipuAI
File "/home/dengkaibiao/anaconda3/lib/python3.10/site-packages/zhipuai/_client.py", line 9, in
from .core._http_client import HttpClient, ZHIPUAI_DEFAULT_MAX_RETRIES
File "/home/dengkaibiao/anaconda3/lib/python3.10/site-packages/zhipuai/core/_http_client.py", line 21, in
from ._request_opt import ClientRequestParam, UserRequestInput
File "/home/dengkaibiao/anaconda3/lib/python3.10/site-packages/zhipuai/core/_request_opt.py", line 25, in
class ClientRequestParam(pydantic.BaseModel):
File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.new
File "pydantic/fields.py", line 504, in pydantic.fields.ModelField.infer
File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.init
File "pydantic/fields.py", line 550, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 661, in pydantic.fields.ModelField._type_analysis
File "pydantic/fields.py", line 806, in pydantic.fields.ModelField._create_sub_type
File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.init
File "pydantic/fields.py", line 550, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 756, in pydantic.fields.ModelField._type_analysis
File "pydantic/fields.py", line 806, in pydantic.fields.ModelField._create_sub_type
File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.init
File "pydantic/fields.py", line 550, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 661, in pydantic.fields.ModelField._type_analysis
File "pydantic/fields.py", line 806, in pydantic.fields.ModelField._create_sub_type
File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.init
File "pydantic/fields.py", line 550, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 753, in pydantic.fields.ModelField._type_analysis
TypeError: Fields of type "<class 'typing.IO'>" are not supported.

Other information

No response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.