Reproducing InternVL3-38B results on MMVet and AI2D

I am running VLMEvalkit on MMVet and AI2D with the following command:

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
export USE_COT="1"
torchrun --master_port=9989 --nproc-per-node=8 run.py --data MMVet AI2D_TEST --model InternVL3-38B

But I am not able to reproduce the result from either the paper or the VLMEvalkit leaderboard. I am getting 76.5 while VLMEvalkit reports 81.1 for MMVet. For AI2D, I get 84.5 vs the reported 88.7. I have tried running multiple times but the variance is pretty low. I am able to perfectly reproduce the results on MME and OCRBench, with a slightly deviated but acceptable performance on HallusionBench. Here is the environment info:

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
accelerate 1.6.0 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
altair 5.5.0 pypi_0 pypi
annotated-types 0.7.0 pypi_0 pypi
anyio 4.9.0 pypi_0 pypi
aom 3.5.0 h27087fc_0 conda-forge
argon2-cffi 23.1.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
arrow 1.3.0 pypi_0 pypi
arxiv 2.2.0 pypi_0 pypi
asttokens 3.0.0 pyhd8ed1ab_1 conda-forge
async-lru 2.0.5 pypi_0 pypi
attrs 25.3.0 pypi_0 pypi
babel 2.17.0 pypi_0 pypi
beautifulsoup4 4.13.4 pypi_0 pypi
bitsandbytes 0.45.5 pypi_0 pypi
blas 1.0 mkl
bleach 6.2.0 pypi_0 pypi
blinker 1.9.0 pypi_0 pypi
brotli-python 1.0.9 py310h6a678d5_9
bzip2 1.0.8 h4bc722e_7 conda-forge
ca-certificates 2025.4.26 hbd8a1cb_0 conda-forge
cachetools 5.5.2 pypi_0 pypi
certifi 2025.4.26 pyhd8ed1ab_0 conda-forge
cffi 1.17.1 pypi_0 pypi
charset-normalizer 3.3.2 pyhd3eb1b0_0
click 8.2.0 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
comm 0.2.2 pyhd8ed1ab_1 conda-forge
contourpy 1.3.2 pypi_0 pypi
cuda-cudart 12.4.127 0 nvidia
cuda-cupti 12.4.127 0 nvidia
cuda-libraries 12.4.1 0 nvidia
cuda-nvrtc 12.4.127 0 nvidia
cuda-nvtx 12.4.127 0 nvidia
cuda-opencl 12.9.19 0 nvidia
cuda-runtime 12.4.1 0 nvidia
cuda-version 12.9 3 nvidia
cycler 0.12.1 pypi_0 pypi
debugpy 1.8.14 py310hf71b8c6_0 conda-forge
decorator 5.2.1 pyhd8ed1ab_0 conda-forge
decord 0.6.0 pypi_0 pypi
deepspeed 0.16.7 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
distro 1.9.0 pypi_0 pypi
docstring-parser 0.16 pypi_0 pypi
dotenv 0.9.9 pypi_0 pypi
duckduckgo-search 5.3.1b1 pypi_0 pypi
einops 0.8.1 pypi_0 pypi
et-xmlfile 2.0.0 pypi_0 pypi
exceptiongroup 1.3.0 pyhd8ed1ab_0 conda-forge
executing 2.2.0 pyhd8ed1ab_0 conda-forge
fastjsonschema 2.21.1 pypi_0 pypi
feedparser 6.0.11 pypi_0 pypi
ffmpeg 4.4.2 gpl_h8dda1f0_112 conda-forge
filelock 3.17.0 py310h06a4308_0
flash-attn 2.7.0.post2 pypi_0 pypi
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.15.0 h7e30c49_1 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.58.0 pypi_0 pypi
fqdn 1.5.1 pypi_0 pypi
freetype 2.13.3 ha770c72_1 conda-forge
func-timeout 4.3.5 pypi_0 pypi
giflib 5.2.2 h5eee18b_0
gmp 6.3.0 h6a678d5_0
gmpy2 2.2.1 py310h5eee18b_0
gnutls 3.7.9 hb077bed_0 conda-forge
griffe 0.49.0 pypi_0 pypi
h11 0.16.0 pypi_0 pypi
h2 4.2.0 pypi_0 pypi
hjson 3.1.0 pypi_0 pypi
hpack 4.1.0 pypi_0 pypi
httpcore 1.0.9 pypi_0 pypi
httpx 0.28.1 pypi_0 pypi
huggingface-hub 0.31.2 pypi_0 pypi
hyperframe 6.1.0 pypi_0 pypi
icu 75.1 he02047a_0 conda-forge
idna 3.7 py310h06a4308_0
imageio 2.37.0 pypi_0 pypi
importlib-metadata 8.6.1 pyha770c72_0 conda-forge
intel-openmp 2021.4.0 h06a4308_3561
ipykernel 6.29.5 pyh3099207_0 conda-forge
ipython 8.36.0 pyh907856f_0 conda-forge
ipywidgets 8.1.7 pypi_0 pypi
isoduration 20.11.0 pypi_0 pypi
jedi 0.19.2 pyhd8ed1ab_1 conda-forge
jinja2 3.1.6 py310h06a4308_0
jiter 0.9.0 pypi_0 pypi
joblib 1.5.0 pypi_0 pypi
jpeg 9e h5eee18b_3
json5 0.12.0 pypi_0 pypi
jsonpointer 3.0.0 pypi_0 pypi
jsonschema 4.23.0 pypi_0 pypi
jsonschema-specifications 2025.4.1 pypi_0 pypi
jupyter 1.1.1 pypi_0 pypi
jupyter-console 6.6.3 pypi_0 pypi
jupyter-events 0.12.0 pypi_0 pypi
jupyter-lsp 2.2.5 pypi_0 pypi
jupyter-server 2.16.0 pypi_0 pypi
jupyter-server-terminals 0.5.3 pypi_0 pypi
jupyter_client 8.6.3 pyhd8ed1ab_1 conda-forge
jupyter_core 5.7.2 pyh31011fe_1 conda-forge
jupyterlab 4.4.2 pypi_0 pypi
jupyterlab-pygments 0.3.0 pypi_0 pypi
jupyterlab-server 2.27.3 pypi_0 pypi
jupyterlab-widgets 3.0.15 pypi_0 pypi
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.8 pypi_0 pypi
krb5 1.21.3 h659f571_0 conda-forge
lagent 0.2.4 pypi_0 pypi
lame 3.100 h7b6447c_0
lazy-loader 0.4 pypi_0 pypi
lcms2 2.15 hfd0df8a_0 conda-forge
ld_impl_linux-64 2.43 h712a8e2_4 conda-forge
lerc 4.0.0 h6a678d5_0
libcublas 12.4.5.8 0 nvidia
libcufft 11.2.1.3 0 nvidia
libcufile 1.14.0.30 4 nvidia
libcurand 10.3.10.19 0 nvidia
libcusolver 11.6.1.9 0 nvidia
libcusparse 12.3.1.170 0 nvidia
libdeflate 1.17 h0b41bf4_0 conda-forge
libdrm 2.4.124 hb9d3cd8_0 conda-forge
libedit 3.1.20250104 pl5321h7949ede_0 conda-forge
libexpat 2.7.0 h5888daf_0 conda-forge
libffi 3.4.6 h2dba641_1 conda-forge
libfreetype 2.13.3 ha770c72_1 conda-forge
libfreetype6 2.13.3 h48d6fc4_1 conda-forge
libgcc 15.1.0 h767d61c_2 conda-forge
libgcc-ng 15.1.0 h69a702a_2 conda-forge
libgomp 15.1.0 h767d61c_2 conda-forge
libiconv 1.18 h4ce23a2_1 conda-forge
libidn2 2.3.4 h5eee18b_0
libjpeg-turbo 2.0.0 h9bf148f_0 pytorch
liblzma 5.8.1 hb9d3cd8_1 conda-forge
liblzma-devel 5.8.1 hb9d3cd8_1 conda-forge
libnpp 12.2.5.30 0 nvidia
libnsl 2.0.1 hd590300_0 conda-forge
libnvfatbin 12.9.19 0 nvidia
libnvjitlink 12.4.127 0 nvidia
libnvjpeg 12.3.1.117 0 nvidia
libpciaccess 0.18 hd590300_0 conda-forge
libpng 1.6.47 h943b412_0 conda-forge
libsodium 1.0.20 h4ab18f5_0 conda-forge
libsqlite 3.49.2 hee588c1_0 conda-forge
libstdcxx 15.1.0 h8f9b012_2 conda-forge
libstdcxx-ng 15.1.0 h4852527_2 conda-forge
libtasn1 4.19.0 h5eee18b_0
libtiff 4.5.0 h6adf6a1_2 conda-forge
libunistring 0.9.10 h27cfd23_0
libuuid 2.38.1 h0b41bf4_0 conda-forge
libva 2.18.0 h0b41bf4_0 conda-forge
libvpx 1.11.0 h9c3ff4c_3 conda-forge
libwebp 1.2.4 h1daa5a0_1 conda-forge
libwebp-base 1.2.4 h166bdaf_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxml2 2.13.8 h4bc477f_0 conda-forge
libzlib 1.3.1 hb9d3cd8_2 conda-forge
llvm-openmp 15.0.7 h0cdce71_0 conda-forge
lz4-c 1.9.4 h6a678d5_1
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 3.0.2 py310h5eee18b_0
matplotlib 3.10.3 pypi_0 pypi
matplotlib-inline 0.1.7 pyhd8ed1ab_1 conda-forge
mdurl 0.1.2 pypi_0 pypi
mistune 3.1.3 pypi_0 pypi
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py310ha2c4b55_0 conda-forge
mkl_fft 1.3.1 py310h2b4bcf5_1 conda-forge
mkl_random 1.2.2 py310h00e6091_0
mmengine 0.10.7 pypi_0 pypi
mpc 1.3.1 h5eee18b_0
mpfr 4.2.1 h5eee18b_0
mpi4py-mpich 3.1.5 pypi_0 pypi
mpmath 1.3.0 py310h06a4308_0
msgpack 1.1.0 pypi_0 pypi
narwhals 1.39.0 pypi_0 pypi
nbclient 0.10.2 pypi_0 pypi
nbconvert 7.16.6 pypi_0 pypi
nbformat 5.10.4 pypi_0 pypi
ncurses 6.5 h2d0b736_3 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_1 conda-forge
nettle 3.9.1 h7ab15ed_0 conda-forge
networkx 3.4.2 py310h06a4308_0
ninja 1.11.1.4 pypi_0 pypi
nltk 3.9.1 pypi_0 pypi
notebook 7.4.2 pypi_0 pypi
notebook-shim 0.2.4 pypi_0 pypi
numpy 1.24.3 py310hd5efca6_0
numpy-base 1.24.3 py310h8e6c178_0
nvidia-ml-py 12.575.51 pypi_0 pypi
ocl-icd 2.3.2 h5eee18b_1
openai 1.78.1 pypi_0 pypi
opencv-python 4.11.0.86 pypi_0 pypi
openh264 2.3.1 hcb278e6_2 conda-forge
openjpeg 2.5.0 hfec8fc6_2 conda-forge
openpyxl 3.1.5 pypi_0 pypi
openssl 3.5.0 h7b32b05_1 conda-forge
overrides 7.7.0 pypi_0 pypi
p11-kit 0.24.1 hc5aa10d_0 conda-forge
packaging 24.2 pypi_0 pypi
pandas 2.2.3 pypi_0 pypi
pandocfilters 1.5.1 pypi_0 pypi
parso 0.8.4 pyhd8ed1ab_1 conda-forge
peft 0.4.0 pypi_0 pypi
pexpect 4.9.0 pyhd8ed1ab_1 conda-forge
phx-class-registry 4.1.0 pypi_0 pypi
pickleshare 0.7.5 pyhd8ed1ab_1004 conda-forge
pillow 11.2.1 pypi_0 pypi
pip 25.1.1 pyh8b19718_0 conda-forge
platformdirs 4.3.8 pyhe01879c_0 conda-forge
prometheus-client 0.21.1 pypi_0 pypi
prompt-toolkit 3.0.51 pyha770c72_0 conda-forge
psutil 7.0.0 py310ha75aee5_0 conda-forge
pthread-stubs 0.4 hb9d3cd8_1002 conda-forge
ptyprocess 0.7.0 pyhd8ed1ab_1 conda-forge
pure_eval 0.2.3 pyhd8ed1ab_1 conda-forge
py-cpuinfo 9.0.0 pypi_0 pypi
pyarrow 20.0.0 pypi_0 pypi
pycparser 2.22 pypi_0 pypi
pydantic 2.11.4 pypi_0 pypi
pydantic-core 2.33.2 pypi_0 pypi
pydeck 0.9.1 pypi_0 pypi
pygments 2.19.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.2.3 pypi_0 pypi
pysocks 1.7.1 py310h06a4308_0
python 3.10.17 hd6af730_0_cpython conda-forge
python-dateutil 2.9.0.post0 pyhff2d567_1 conda-forge
python-dotenv 1.1.0 pypi_0 pypi
python-json-logger 3.3.0 pypi_0 pypi
python_abi 3.10 7_cp310 conda-forge
pytorch 2.5.1 py3.10_cuda12.4_cudnn9.1.0_0 pytorch
pytorch-cuda 12.4 hc786d27_7 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2025.2 pypi_0 pypi
pyyaml 6.0.2 py310h5eee18b_0
pyzmq 26.4.0 py310h71f11fc_0 conda-forge
readline 8.2 h8c095d6_2 conda-forge
referencing 0.36.2 pypi_0 pypi
requests 2.32.3 py310h06a4308_1
rfc3339-validator 0.1.4 pypi_0 pypi
rfc3986-validator 0.1.1 pypi_0 pypi
rich 14.0.0 pypi_0 pypi
rpds-py 0.24.0 pypi_0 pypi
safetensors 0.5.3 pypi_0 pypi
scikit-image 0.25.2 pypi_0 pypi
scipy 1.15.3 pypi_0 pypi
send2trash 1.8.3 pypi_0 pypi
sentencepiece 0.2.0 pypi_0 pypi
setuptools 80.1.0 pyhff2d567_0 conda-forge
sgmllib3k 1.0.0 pypi_0 pypi
shtab 1.7.2 pypi_0 pypi
six 1.17.0 pyhd8ed1ab_0 conda-forge
sniffio 1.3.1 pypi_0 pypi
socksio 1.0.0 pypi_0 pypi
soupsieve 2.7 pypi_0 pypi
sqlite 3.49.2 h9eae976_0 conda-forge
stack_data 0.6.3 pyhd8ed1ab_1 conda-forge
streamlit 1.45.1 pypi_0 pypi
sty 1.0.6 pypi_0 pypi
svt-av1 1.4.1 hcb278e6_0 conda-forge
sympy 1.13.1 pypi_0 pypi
tbb 2021.8.0 hdb19cb5_0
tenacity 9.1.2 pypi_0 pypi
terminado 0.18.1 pypi_0 pypi
tifffile 2025.5.10 pypi_0 pypi
tiktoken 0.9.0 pypi_0 pypi
timeout-decorator 0.5.0 pypi_0 pypi
timm 1.0.15 pypi_0 pypi
tinycss2 1.4.0 pypi_0 pypi
tk 8.6.13 noxft_h4845f30_101 conda-forge
tokenizers 0.20.3 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
tomli 2.2.1 pypi_0 pypi
torchaudio 2.5.1 py310_cu124 pytorch
torchtriton 3.1.0 py310 pytorch
torchvision 0.20.1 py310_cu124 pytorch
tornado 6.4.2 py310ha75aee5_0 conda-forge
tqdm 4.67.1 pypi_0 pypi
traitlets 5.14.3 pyhd8ed1ab_1 conda-forge
transformers 4.45.1 pypi_0 pypi
transformers-stream-generator 0.0.5 pypi_0 pypi
trl 0.10.1 pypi_0 pypi
typeguard 4.4.2 pypi_0 pypi
types-python-dateutil 2.9.0.20241206 pypi_0 pypi
typing-inspection 0.4.0 pypi_0 pypi
typing_extensions 4.13.2 pyh29332c3_0 conda-forge
tyro 0.9.22 pypi_0 pypi
tzdata 2025b h78e105d_0 conda-forge
uri-template 1.3.0 pypi_0 pypi
urllib3 2.3.0 py310h06a4308_0
validators 0.35.0 pypi_0 pypi
watchdog 6.0.0 pypi_0 pypi
wcwidth 0.2.13 pyhd8ed1ab_1 conda-forge
webcolors 24.11.1 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.8.0 pypi_0 pypi
wheel 0.45.1 pyhd8ed1ab_1 conda-forge
widgetsnbextension 4.0.14 pypi_0 pypi
x264 1!164.3095 h166bdaf_2 conda-forge
x265 3.5 h924138e_3 conda-forge
xlsxwriter 3.2.3 pypi_0 pypi
xorg-fixesproto 5.0 hb9d3cd8_1003 conda-forge
xorg-kbproto 1.0.7 hb9d3cd8_1003 conda-forge
xorg-libx11 1.8.4 h0b41bf4_0 conda-forge
xorg-libxau 1.0.12 hb9d3cd8_0 conda-forge
xorg-libxdmcp 1.1.5 hb9d3cd8_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-xextproto 7.3.0 hb9d3cd8_1004 conda-forge
xorg-xproto 7.0.31 hb9d3cd8_1008 conda-forge
xtuner 0.1.23 pypi_0 pypi
xz 5.8.1 hbcc6ac9_1 conda-forge
xz-gpl-tools 5.8.1 hbcc6ac9_1 conda-forge
xz-tools 5.8.1 hb9d3cd8_1 conda-forge
yaml 0.2.5 h7b6447c_0
yapf 0.43.0 pypi_0 pypi
zeromq 4.3.5 h3b0a872_7 conda-forge
zipp 3.21.0 pyhd8ed1ab_1 conda-forge
zlib 1.3.1 hb9d3cd8_2 conda-forge
zstd 1.5.7 hb8e6e7a_2 conda-forge

Start with the reported torchrun command in run.py using InternVL3-38B, then compare the MMVet and AI2D_TEST scores against the stated leaderboard values. Check the provided Python, PyTorch, CUDA, Transformers, and VLMEvalKit environment details; done means identifying why the two results differ or documenting a reproducible explanation.