lmstudio-ai / lmstudio-ai/docs
Feature request: Multiple server runtimes
- Dominant language
- MDX
- Stars
- 172
- Forks
- 128
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
I have multiple GPUs of different architectures in the same machine (for example, a laptop with an intel iGPU + nvidia dGPU), where I want to run Vulkan based llama.cpp in iGPU (where I have 96 Gb of shared memory, but it is awfully slow) for one model, and cuda based llama in dGPU (where I have only 4 GB of memory, and I can only feasibly run nemotron-4b for light tasks)
I would be very glad if I could do this within the lms.
Maybe the selection of runtime can be carried to model load, and can be selected per model instead of a global setting, sparkrun style?
Contributor guide
Research direction
The issue identifies no files, tests, or entry points, so first map where server runtime selection and per-model loading are documented or implemented. Confirm the supported runtime behavior and define completion as a documented, working way to choose different runtimes for different models.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100