NVIDIA / NVIDIA/Personal-AI-Router
[Feature]: Multiple ollama instances on single host support
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 250
- Avg merge
- 23h 27m
- Merged PRs (30d)
- 1
Description
Area
Routing and scheduling
User problem
I have a system with two GPUs from different vendors, one being a nvidia GPU. I've had to setup two ollama instances locally on different ports to get this to work with different settings. I'd like to be able to expose both ollama instances to the cluster
Desired outcome
Both ollama instances show up for a node (or it appears as two nodes)
Alternatives considered
No response
Compatibility and security implications
No response
Validation approach
Launch two ollama instances on different ports, ideally with specific GPUs enabled for each
Confirmations
- I searched existing issues for duplicates.
- I agree to follow the Code of Conduct.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the routing and scheduling entry points that discover and expose Ollama instances. Trace how a node's Ollama endpoint and configuration are represented, then determine whether two instances should be exposed as separate nodes or as endpoints on one node. Validate by launching two Ollama instances on different ports with separate GPU settings and confirming both appear in the cluster.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, ollama
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100