vllm-project / vllm-project/aibrix
Pluggable Gateway
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
I am a [kgateway](https://kgateway.dev/) maintainer. In the upcoming release, kgateway will support Kubernetes [Gateway API Inference Extension](https://gateway-api-inference-extension.sigs.k8s.io/). The aibrix [Gateway](https://aibrix.readthedocs.io/latest/features/gateway-plugins.html) overlaps with Gateway API Inference Extension, specifically the endpoint picker extension. Can the aibrix Gateway component become extensible in the [architecture](https://aibrix.readthedocs.io/latest/designs/architecture.html)? A spec, conformance tests, etc. can be defined to support aibrix Gateway extensibility. By doing so, aibrix will benefit from wider adoption.
### Use Case
As a kgateway user with Gateway API inference Extension, I prefer to integrate with Aibrix instead of having separate Gateways.
### Proposed Solution
Develop an interface, spec, conformance tests, etc. to make the aibrix extensible and allow integration with Gateway API inference extension conformant implementations.
Contributor guide
Research direction
Start with the Gateway component and the linked architecture and gateway-plugins documentation, then review the Gateway API Inference Extension requirements, especially the endpoint picker extension. The work is done when an agreed extensibility interface and specification exist with conformance tests that support conformant integrations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100