InternLM / InternLM/lmdeploy

[Bug] 使用v0.10.1版本部署internvl2.5-38b-mpo-awq报错,如何解决?

Open
#4,040 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Checklist

- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

/home/li_mingze/.local/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]
Traceback (most recent call last):
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2302, in __getattr__
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2332, in _get_module
raise e
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2330, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1381, in _gcd_import
File "", line 1354, in _find_and_load
File "", line 1325, in _find_and_load_unlocked
File "", line 929, in _load_unlocked
File "", line 994, in exec_module
File "", line 488, in _call_with_frames_removed
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/models/auto/processing_auto.py", line 28, in
from ...processing_utils import ProcessorMixin
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/processing_utils.py", line 34, in
from .audio_utils import load_audio
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/audio_utils.py", line 42, in
import soundfile as sf
File "/home/li_mingze/.local/lib/python3.12/site-packages/soundfile.py", line 17, in
from _soundfile import ffi as _ffi
File "/home/li_mingze/.local/lib/python3.12/site-packages/_soundfile.py", line 2, in
import _cffi_backend
ModuleNotFoundError: No module named '_cffi_backend'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/bin/lmdeploy", line 7, in
sys.exit(run())
^^^^^
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/entrypoint.py", line 39, in run
args.run(args)
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/serve.py", line 256, in api_server
run_api_server(args.model_path,
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/serve/openai/api_server.py", line 1286, in serve
_, pipeline_class = get_task(model_path)
^^^^^^^^^^^^^^^^^^^^
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/archs.py", line 136, in get_task
from lmdeploy.serve.vl_async_engine import VLAsyncEngine
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/serve/vl_async_engine.py", line 12, in
from lmdeploy.vl.engine import ImageEncoder
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/vl/engine.py", line 11, in
from lmdeploy.vl.model.builder import load_vl_model
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/vl/model/builder.py", line 15, in
from .gemma3_vl import Gemma3VisionModel # noqa F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/site-packages/lmdeploy/vl/model/gemma3_vl.py", line 5, in
from transformers import AutoConfig, AutoProcessor
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2305, in __getattr__
raise ModuleNotFoundError(
ModuleNotFoundError: Could not import module 'AutoProcessor'. Are this object's requirements defined correctly?

### Reproduction

CUDA_VISIBLE_DEVICES=0,1 lmdeploy serve api_server /data3/mingze/models/OpenGVLab/InternVL2_5-38B-MPO-AWQ --server-port 2556 --cache-max-entry-count 0.5 --tp 2 --backend pytorch > /data3/mi
ngze/big_test/38_lmdeploy.log 2>&1 &

### Environment

```Shell
Package Version
--------------------------------- -------------
accelerate 1.10.1
addict 2.4.0
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
annotated-types 0.7.0
anyio 4.10.0
astor 0.8.1
attrs 25.3.0
blake3 1.0.5
cachetools 6.2.0
cbor2 5.7.0
certifi 2025.8.3
charset-normalizer 3.4.3
click 8.2.1
cloudpickle 3.1.1
compressed-tensors 0.11.0
cupy-cuda12x 13.6.0
depyf 0.19.0
dill 0.4.0
diskcache 5.6.3
distro 1.9.0
dnspython 2.8.0
einops 0.8.1
email-validator 2.3.0
fastapi 0.116.2
fastapi-cli 0.0.11
fastapi-cloud-cli 0.1.5
fastrlock 0.8.3
filelock 3.19.1
fire 0.7.1
frozenlist 1.7.0
fsspec 2025.9.0
genson 1.3.0
gguf 0.17.1
h11 0.16.0
hf-xet 1.1.10
httpcore 1.0.9
httptools 0.6.4
httpx 0.28.1
huggingface-hub 0.35.0
idna 3.10
interegular 0.3.3
Jinja2 3.1.6
jiter 0.11.0
jsonpath-ng 1.7.0
jsonschema 4.25.1
jsonschema-specifications 2025.9.1
lark 1.2.2
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.11.3
lmdeploy 0.10.1
markdown-it-py 4.0.0
MarkupSafe 3.0.2
mdurl 0.1.2
mistral_common 1.8.5
mmengine-lite 0.10.7
mpmath 1.3.0
msgpack 1.1.1
msgspec 0.19.0
multidict 6.6.4
networkx 3.5
ninja 1.13.0
numba 0.61.2
numpy 2.2.6
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-ml-py 13.580.82
nvidia-nccl-cu12 2.27.3
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvtx-cu12 12.8.90
nvitop 1.5.3
openai 1.107.3
openai-harmony 0.0.4
opencv-python-headless 4.12.0.88
outlines 1.2.5
outlines_core 0.2.11
packaging 25.0
partial-json-parser 0.2.1.1.post6
peft 0.14.0
pillow 11.3.0
pip 25.2
platformdirs 4.4.0
ply 3.11
prometheus_client 0.22.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.3.2
protobuf 6.32.1
psutil 7.0.0
py-cpuinfo 9.0.0
pybase64 1.4.2
pycountry 24.6.1
pydantic 2.11.9
pydantic_core 2.33.2
pydantic-extra-types 2.10.5
Pygments 2.19.2
pynvml 13.0.1
python-dotenv 1.1.1
python-json-logger 3.3.0
python-multipart 0.0.20
PyYAML 6.0.2
pyzmq 27.1.0
ray 2.49.1
referencing 0.36.2
regex 2025.9.1
requests 2.32.5
rich 14.1.0
rich-toolkit 0.15.1
rignore 0.6.4
rpds-py 0.27.1
safetensors 0.6.2
scipy 1.16.2
sentencepiece 0.2.1
sentry-sdk 2.38.0
setproctitle 1.3.7
setuptools 79.0.1
shellingham 1.5.4
shortuuid 1.0.13
sniffio 1.3.1
soundfile 0.13.1
soxr 1.0.0
starlette 0.48.0
sympy 1.14.0
termcolor 3.1.0
tiktoken 0.11.0
timm 1.0.19
tokenizers 0.22.0
torch 2.8.0
torchaudio 2.8.0
torchvision 0.23.0
tqdm 4.67.1
transformers 4.56.1
triton 3.4.0
typer 0.17.4
typing_extensions 4.15.0
typing-inspection 0.4.1
urllib3 2.5.0
uvicorn 0.35.0
uvloop 0.21.0
vllm 0.10.2
watchfiles 1.1.0
websockets 15.0.1
wheel 0.45.1
xformers 0.0.32.post1
xgrammar 0.1.23
yapf 0.43.0
yarl 1.20.1
```

### Error traceback

```Shell

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.