vllm-project / vllm-project/aibrix
[Ecosystem] NIM inference with AIBrix
Open
kind/support
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
I’m not entirely certain if this aligns with our goals. Some potential users are looking to integrate NIM (NVIDIA Inference Manager), which is already supported in Kserve. We might need to evaluate whether this fits into our roadmap.
- https://github.com/NVIDIA/nim-deploy/blob/main/kserve/nim-models/README.md
- https://blogs.nvidia.com/blog/kserve-nim-inference/
### Use Case
Using NIM with AIBrix
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.