[Bug] Issues Running Vision Language Models in Docker
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
Hi folks, thanks for this great project. I am raising this issue in case it helps anyone else with Docker.
When running different models with Docker as per the instructions at [Option 2: Deploying with Docker](https://lmdeploy.readthedocs.io/en/latest/serving/api_server_vl.html#option-2-deploying-with-docker), I have run into various issues with Python dependencies. This has been resolved by creating a new `Dockerfile` with the necessary Python deps installed. For example, when running Llava 1.6 34B one runs into dependency issues such as no such module `timm`, and "install llava at git@...":
The following Dockerfile works for me for Llava:
```Docker
FROM openmmlab/lmdeploy:latest
RUN apt-get update && apt-get install -y python3 python3-pip git
WORKDIR /app
RUN pip3 install --upgrade pip
RUN pip3 install timm
RUN pip3 install git+https://github.com/haotian-liu/LLaVA.git --no-deps
COPY . .
CMD ["lmdeploy", "serve", "api_server", "liuhaotian/llava-v1.6-34b"]
```
Likewise for Yi-VL.
For Deepseek-VL, this worked:
```Docker
FROM openmmlab/lmdeploy:latest
RUN apt-get update && apt-get install -y python3 python3-pip git
WORKDIR /app
RUN pip3 install --upgrade pip
RUN pip3 install git+https://github.com/deepseek-ai/DeepSeek-VL.git --no-deps
RUN pip3 install attrdict
RUN pip3 install timm
COPY . .
CMD ["lmdeploy", "serve", "api_server", "deepseek-ai/deepseek-vl-7b-chat"]
```
### Reproduction
```bash
docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=" \
-p 23333:23333 \
--ipc=host \
openmmlab/lmdeploy:latest \
lmdeploy serve api_server liuhaotian/llava-v1.6-34b
```
### Environment
```Shell
Ubuntu 22.04
```
### Error traceback
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.