lmstudio-ai / lmstudio-ai/docs

Feature request: Multiple server runtimes

Open
#238 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
172
Forks
128
Avg merge
8m
Merged PRs (30d)
1

Description

I have multiple GPUs of different architectures in the same machine (for example, a laptop with an intel iGPU + nvidia dGPU), where I want to run Vulkan based llama.cpp in iGPU (where I have 96 Gb of shared memory, but it is awfully slow) for one model, and cuda based llama in dGPU (where I have only 4 GB of memory, and I can only feasibly run nemotron-4b for light tasks)
I would be very glad if I could do this within the lms.
Maybe the selection of runtime can be carried to model load, and can be selected per model instead of a global setting, sparkrun style?

Contributor guide

Open the contributing guide

Research direction

The issue identifies no files, tests, or entry points, so first map where server runtime selection and per-model loading are documented or implemented. Confirm the supported runtime behavior and define completion as a documented, working way to choose different runtimes for different models.

Written by the indexing model from the issue text.

Assessment

Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.