InternLM / InternLM/lmdeploy

[Feature] Create Cuda 12 docker images

Open
#1,709 5 comments 0 reactions 1 assignee Claimed by @lvhan028 View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

Create container images for CUDA 12+ versions. Currently to run a cuda-12 container image one must build their own and edit the dockerfile.

### Related resources

I propose adding a new dockerfile for specific cuda versions:
[docker/Dockerfile](https://github.com/InternLM/lmdeploy/blob/main/docker/Dockerfile)

```diff
# CUDA 12.3 example --
# Uses system python version
- FROM nvcr.io/nvidia/tritonserver:22.12-py3
+ FROM nvcr.io/nvidia/tritonserver:24.04-py3

RUN rm /etc/apt/sources.list.d/cuda*.list && apt-get update && apt-get install -y --no-install-recommends \
- rapidjson-dev libgoogle-glog-dev gdb python3.8-venv \
+ rapidjson-dev libgoogle-glog-dev gdb python3-venv \
- && rm -rf /var/lib/apt/lists/* && cd /opt && python3 -m venv py38
+ && rm -rf /var/lib/apt/lists/* && cd /opt && python3 -m venv py3

- ENV PATH=/opt/py38/bin:$PATH
+ ENV PATH=/opt/py3/bin:$PATH

RUN python3 -m pip install --no-cache-dir --upgrade pip &&\
python3 -m pip install --no-cache-dir torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cu121 &&\
python3 -m pip install --no-cache-dir cmake packaging wheel &&\
+ python3 -m pip install --no-cache-dir timm flash-attn # For Quality of Life
```

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.