vllm-project / vllm-project/production-stack

bug: lmcache/vllm-openai for RTX 5090

Open
#528 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
2.6k
Forks
503
Avg merge
4d 17h
Merged PRs (30d)
8

Description

Describe the bug

What version of lmcache/vllm-openai should I take to run on RTX 5090?

I'm trying to run production-stack in k3s and get the error:

NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

I have already tried many versions of the image, but none of them worked on the RTX 5090.

To Reproduce

Driver Version: 570.133.07
CUDA Version: 12.8

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0

$ python3 -c "
import torch
print(f'PyTorch version: {torch.version}')
print(f'CUDA available: {torch.cuda.is_available()}')
print(f'CUDA version in PyTorch: {torch.version.cuda}')
print(f'cuDNN version: {torch.backends.cudnn.version()}')
if torch.cuda.is_available():
print(f'Current GPU: {torch.cuda.get_device_name(torch.cuda.current_device())}')
"
PyTorch version: 2.7.1+cu128
CUDA available: False
CUDA version in PyTorch: 12.8
cuDNN version: 90701

Expected behavior

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking the lmcache/vllm-openai image versions used by production-stack and compare their PyTorch CUDA support with the reported RTX 5090, driver, and CUDA 12.8 details. Reproduce the deployment in k3s and verify whether a compatible image exists; done means identifying and documenting a working version or the required compatibility change.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python, pytorch
Domain
devops, infrastructure, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.