vllm-project / vllm-project/production-stack
bug: lmcache/vllm-openai for RTX 5090
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the bug
What version of lmcache/vllm-openai should I take to run on RTX 5090?
I'm trying to run production-stack in k3s and get the error:
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
I have already tried many versions of the image, but none of them worked on the RTX 5090.
To Reproduce
Driver Version: 570.133.07
CUDA Version: 12.8
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0
$ python3 -c "
import torch
print(f'PyTorch version: {torch.version}')
print(f'CUDA available: {torch.cuda.is_available()}')
print(f'CUDA version in PyTorch: {torch.version.cuda}')
print(f'cuDNN version: {torch.backends.cudnn.version()}')
if torch.cuda.is_available():
print(f'Current GPU: {torch.cuda.get_device_name(torch.cuda.current_device())}')
"
PyTorch version: 2.7.1+cu128
CUDA available: False
CUDA version in PyTorch: 12.8
cuDNN version: 90701
Expected behavior
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by checking the lmcache/vllm-openai image versions used by production-stack and compare their PyTorch CUDA support with the reported RTX 5090, driver, and CUDA 12.8 details. Reproduce the deployment in k3s and verify whether a compatible image exists; done means identifying and documenting a working version or the required compatibility change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python, pytorch
- Domain
- devops, infrastructure, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100