[Feature] Create Cuda 12 docker images
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
Create container images for CUDA 12+ versions. Currently to run a cuda-12 container image one must build their own and edit the dockerfile.
### Related resources
I propose adding a new dockerfile for specific cuda versions:
[docker/Dockerfile](https://github.com/InternLM/lmdeploy/blob/main/docker/Dockerfile)
```diff
# CUDA 12.3 example --
# Uses system python version
- FROM nvcr.io/nvidia/tritonserver:22.12-py3
+ FROM nvcr.io/nvidia/tritonserver:24.04-py3
RUN rm /etc/apt/sources.list.d/cuda*.list && apt-get update && apt-get install -y --no-install-recommends \
- rapidjson-dev libgoogle-glog-dev gdb python3.8-venv \
+ rapidjson-dev libgoogle-glog-dev gdb python3-venv \
- && rm -rf /var/lib/apt/lists/* && cd /opt && python3 -m venv py38
+ && rm -rf /var/lib/apt/lists/* && cd /opt && python3 -m venv py3
- ENV PATH=/opt/py38/bin:$PATH
+ ENV PATH=/opt/py3/bin:$PATH
RUN python3 -m pip install --no-cache-dir --upgrade pip &&\
python3 -m pip install --no-cache-dir torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cu121 &&\
python3 -m pip install --no-cache-dir cmake packaging wheel &&\
+ python3 -m pip install --no-cache-dir timm flash-attn # For Quality of Life
```
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.