open-compass / open-compass/VLMEvalKit
lmdeploy support Qwen3-VL-4B-Instruct
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 17
Description
commands:
lmdeploy serve api_server Qwen/Qwen3-VL-4B-Instruct --server-port 23333
`Package Version Editable project location
absl-py 2.3.1
accelerate 1.10.1
addict 2.4.0
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
airportsdata 20250909
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
annotated-types 0.7.0
anthropic 0.69.0
antlr4-python3-runtime 4.9.3
anyio 4.11.0
apex 0.1
astor 0.8.1
asttokens 3.0.0
attrdict 2.0.1
attrs 25.3.0
audioread 3.0.1
auto_gptq 0.7.1
autoawq 0.2.9
av 15.1.0
binpacking 1.5.2
bitsandbytes 0.48.0
blake3 1.0.7
blobfile 3.0.0
boto3 1.40.42
botocore 1.40.42
Brotli 1.1.0
build 1.3.0
cachetools 6.2.0
cbor2 5.7.0
certifi 2025.8.3
cffi 2.0.0
cfgv 3.4.0
charset-normalizer 3.4.3
click 8.3.0
cloudpickle 3.1.1
colorama 0.4.6
colorlog 6.9.0
comm 0.2.3
compressed-tensors 0.10.2
contourpy 1.3.3
cpm-kernels 1.0.11
crcmod 1.7
cryptography 46.0.2
cuda-bindings 13.0.1
cuda-pathfinder 1.3.0
cuda-python 13.0.1
cupy-cuda12x 13.6.0
cycler 0.12.1
dacite 1.9.2
datasets 3.6.0
debugpy 1.8.17
decorator 5.2.1
decord 0.6.0
deepspeed 0.17.6
depyf 0.19.0
diffusers 0.35.1
dill 0.3.8
diskcache 5.6.3
distlib 0.4.0
distro 1.9.0
dnspython 2.8.0
docstring_parser 0.17.0
dotenv 0.9.9
e 1.4.5
editdistance 0.8.1
einops 0.8.1
email-validator 2.3.0
et_xmlfile 2.0.0
evalscope 1.0.2
evaluate 0.4.6
executing 2.2.0
fastapi 0.118.0
fastapi-cli 0.0.13
fastapi-cloud-cli 0.2.1
fastrlock 0.8.3
fastuuid 0.13.5
ffmpy 0.6.1
filelock 3.19.1
fire 0.7.1
flash_attn 2.7.4.post1
flashinfer-python 0.2.9rc2
fonttools 4.60.1
frozenlist 1.7.0
fsspec 2025.3.0
funasr 1.2.7
func-timeout 4.3.5
future 1.0.0
fuzzywuzzy 0.18.0
gekko 1.3.0
genson 1.3.0
gguf 0.17.1
gitdb 4.0.12
GitPython 3.1.45
google-auth 2.41.1
google-genai 1.39.1
gradio 5.32.1
gradio_client 1.10.2
groovy 0.1.2
grpcio 1.75.1
h11 0.16.0
h5py 3.14.0
hf_transfer 0.1.9
hf-xet 1.1.10
hjson 3.1.0
httpcore 1.0.9
httptools 0.6.4
httpx 0.28.1
huggingface-hub 0.35.3
human-eval 1.0.3
hydra-core 1.3.2
icecream 2.1.6
identify 2.6.14
idna 3.10
imageio 2.37.0
immutabledict 4.2.1
importlib_metadata 8.7.0
interegular 0.3.3
ipdb 0.13.13
ipykernel 7.0.1
ipython 9.6.0
ipython_pygments_lexers 1.1.1
jaconv 0.4.0
jamo 0.4.1
jedi 0.19.2
jieba 0.42.1
Jinja2 3.1.6
jiter 0.11.0
jmespath 0.10.0
joblib 1.5.2
json_repair 0.51.0
json5 0.12.1
jsonlines 4.0.0
jsonpath-ng 1.7.0
jsonschema 4.25.1
jsonschema-specifications 2025.9.1
jupyter_client 8.6.3
jupyter_core 5.8.1
kaldiio 2.18.1
kiwisolver 1.4.9
langdetect 1.0.9
lark 1.2.2
latex2sympy2_extended 1.0.6
lazy_loader 0.4
Levenshtein 0.27.1
librosa 0.11.0
liger_kernel 0.6.2
lightning-utilities 0.15.2
litellm 1.77.5
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.10.12
lmdeploy 0.10.1
lxml 6.0.2
Markdown 3.9
markdown-it-py 4.0.0
MarkupSafe 3.0.3
math-verify 0.5.2
matplotlib 3.10.6
matplotlib-inline 0.1.7
mdurl 0.1.2
megatron-core 0.13.2
mistral_common 1.8.5
ml_dtypes 0.5.3
mmengine-lite 0.10.7
modelscope 1.29.2
more-itertools 10.8.0
mpmath 1.3.0
ms-agent 1.3.0
ms-opencompass 0.1.6
ms-swift 3.10.0.dev0 /xxx/ms-swift-3.9.0
ms-vlmeval 0.0.18
msgpack 1.1.1
msgspec 0.19.0
multidict 6.6.4
multiprocess 0.70.16
nest-asyncio 1.6.0
networkx 3.5
ninja 1.13.0
nltk 3.9.2
nodeenv 1.9.1
numba 0.61.2
numpy 1.26.4
nvidia-cublas-cu12 12.6.4.1
nvidia-cuda-cupti-cu12 12.6.80
nvidia-cuda-nvrtc-cu12 12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12 9.5.1.17
nvidia-cudnn-frontend 1.14.1
nvidia-cufft-cu12 11.3.0.4
nvidia-cufile-cu12 1.11.1.6
nvidia-curand-cu12 10.3.7.77
nvidia-cusolver-cu12 11.7.1.2
nvidia-cusparse-cu12 12.5.4.2
nvidia-cusparselt-cu12 0.6.3
nvidia-ml-py 13.580.82
nvidia-nccl-cu12 2.26.2
nvidia-nvjitlink-cu12 12.6.85
nvidia-nvshmem-cu12 3.4.5
nvidia-nvtx-cu12 12.6.77
nvitop 1.5.3
omegaconf 2.3.0
onnx 1.19.0
onnx-ir 0.1.10
onnxscript 0.3.1
openai 2.0.0
openai-harmony 0.0.4
openai-whisper 20250625
OpenCC 1.1.9
opencv-python 4.11.0.86
opencv-python-headless 4.12.0.88
openpyxl 3.1.5
optimum 1.27.0
orjson 3.11.3
oss2 2.19.1
outlines 0.1.11
outlines_core 0.1.26
overrides 7.7.0
packaging 25.0
pandas 2.3.2
parso 0.8.5
partial-json-parser 0.2.1.1.post6
peft 0.14.0
pexpect 4.9.0
pillow 11.3.0
pip 23.3.2
platformdirs 4.4.0
ply 3.11
pooch 1.8.2
portalocker 3.2.0
pre_commit 4.3.0
prettytable 3.16.0
prometheus_client 0.23.1
prometheus-fastapi-instrumentator 7.1.0
prompt_toolkit 3.0.52
propcache 0.3.2
protobuf 6.32.1
psutil 7.1.0
ptyprocess 0.7.0
pure_eval 0.2.3
py-cpuinfo 9.0.0
py-spy 0.4.1
pyarrow 21.0.0
pyasn1 0.6.1
pyasn1_modules 0.4.2
pybase64 1.4.2
pybind11 3.0.0
pycountry 24.6.1
pycparser 2.22
pycryptodome 3.23.0
pycryptodomex 3.23.0
pydantic 2.11.9
pydantic_core 2.33.2
pydantic-extra-types 2.10.5
pydub 0.25.1
pyecharts 2.0.8
Pygments 2.19.2
pynndescent 0.5.13
pynvml 13.0.1
pyparsing 3.2.5
pypinyin 0.55.0
pyproject_hooks 1.2.0
python-dateutil 2.9.0.post0
python-dotenv 1.1.1
python-json-logger 3.3.0
python-Levenshtein 0.27.1
python-multipart 0.0.20
pytorch-wpe 0.0.1
pytz 2025.2
PyYAML 6.0.2
pyzmq 27.1.0
qwen-omni-utils 0.0.8
qwen-vl-utils 0.0.14
rank-bm25 0.2.2
RapidFuzz 3.14.1
ray 2.49.2
referencing 0.36.2
regex 2025.9.1
requests 2.32.5
rich 13.9.4
rich-toolkit 0.15.1
rignore 0.6.4
rouge 1.0.1
rouge-chinese 1.0.3
rouge-score 0.1.2
rpds-py 0.27.1
rsa 4.9.1
ruff 0.13.2
s3transfer 0.14.0
sacrebleu 2.5.1
safehttpx 0.1.6
safetensors 0.6.2
scikit-learn 1.7.2
scipy 1.16.2
seaborn 0.13.2
semantic-version 2.10.0
sentence-transformers 5.1.1
sentencepiece 0.2.1
sentry-sdk 2.39.0
setproctitle 1.3.7
setuptools 65.5.1
sgl-kernel 0.2.8
sglang 0.4.10.post2
shellingham 1.5.4
shortuuid 1.0.13
simplejson 3.20.1
six 1.17.0
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
soundfile 0.13.1
soxr 1.0.0
stack-data 0.6.3
starlette 0.48.0
sty 1.0.6
swankit 0.2.4
swanlab 0.6.10
sympy 1.14.0
tabulate 0.9.0
tenacity 9.1.2
tensorboard 2.20.0
tensorboard-data-server 0.7.2
tensorboardX 2.6.4
termcolor 3.1.0
threadpoolctl 3.6.0
tiktoken 0.11.0
timeout-decorator 0.5.0
timm 1.0.16
tokenizers 0.22.1
tomlkit 0.13.3
torch 2.7.1
torch-complex 0.4.4
torch_memory_saver 0.0.8
torchao 0.9.0
torchaudio 2.7.1
torchmetrics 1.8.2
torchvision 0.22.1
tornado 6.5.2
tqdm 4.67.1
traitlets 5.14.3
transformer_engine 2.6.0.post1
transformer_engine_cu12 2.6.0.post1
transformer-engine-torch 2.6.0.post1
transformers 4.57.1
transformers-stream-generator 0.0.5
triton 3.3.1
trl 0.20.0
typer 0.19.2
typing_extensions 4.15.0
typing-inspection 0.4.2
tzdata 2025.2
umap-learn 0.5.9.post2
urllib3 2.5.0
uvicorn 0.37.0
uvloop 0.21.0
validators 0.35.0
virtualenv 20.34.0
vllm 0.10.1.1
vlmeval 0.1.0 /xxx/VLMEvalKit-1019
wandb 0.22.1
watchfiles 1.1.0
wcwidth 0.2.14
websockets 15.0.1
Werkzeug 3.1.3
wheel 0.45.1
word2number 1.1
wrapt 1.17.3
xformers 0.0.31
xgrammar 0.1.22
xlsxwriter 3.2.9
xxhash 3.5.0
yapf 0.43.0
yarl 1.20.1
zipp 3.23.0
zstandard 0.25.0`
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided lmdeploy serve api_server Qwen/Qwen3-VL-4B-Instruct --server-port 23333 command and locate VLMEvalKit’s lmdeploy integration; no repository file or test is named in the issue. Done means Qwen3-VL-4B-Instruct is supported and its serving or evaluation path works, with project-specific validation identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100