vllm-project / vllm-project/aibrix

[Ecosystem] NIM inference with AIBrix

Open
#735 1 comment 1 reaction 0 assignees View on GitHub
kind/support
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

I’m not entirely certain if this aligns with our goals. Some potential users are looking to integrate NIM (NVIDIA Inference Manager), which is already supported in Kserve. We might need to evaluate whether this fits into our roadmap.

- https://github.com/NVIDIA/nim-deploy/blob/main/kserve/nim-models/README.md
- https://blogs.nvidia.com/blog/kserve-nim-inference/

### Use Case

Using NIM with AIBrix

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.