vllm-project / vllm-project/aibrix

Pluggable Gateway

Open
#861 7 comments 4 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 19h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

I am a [kgateway](https://kgateway.dev/) maintainer. In the upcoming release, kgateway will support Kubernetes [Gateway API Inference Extension](https://gateway-api-inference-extension.sigs.k8s.io/). The aibrix [Gateway](https://aibrix.readthedocs.io/latest/features/gateway-plugins.html) overlaps with Gateway API Inference Extension, specifically the endpoint picker extension. Can the aibrix Gateway component become extensible in the [architecture](https://aibrix.readthedocs.io/latest/designs/architecture.html)? A spec, conformance tests, etc. can be defined to support aibrix Gateway extensibility. By doing so, aibrix will benefit from wider adoption.

### Use Case

As a kgateway user with Gateway API inference Extension, I prefer to integrate with Aibrix instead of having separate Gateways.

### Proposed Solution

Develop an interface, spec, conformance tests, etc. to make the aibrix extensible and allow integration with Gateway API inference extension conformant implementations.

Contributor guide

Open the contributing guide

Research direction

Start with the Gateway component and the linked architecture and gateway-plugins documentation, then review the Gateway API Inference Extension requirements, especially the endpoint picker extension. The work is done when an agreed extensibility interface and specification exist with conformance tests that support conformant integrations.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.