vllm-project / vllm-project/aibrix
[Discussion] Simplify AIBrix deployment by removing Envoy Gateway
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🚀 Feature Description and Motivation
Today AIBrix recommends/ships Envoy Gateway, but we typically run a `single` gateway instance. Envoy Gateway adds installation and controller complexity that many users don’t need. Propose an option to deploy vanilla Envoy + ext-proc (External Processing), fully managed by AIBrix (no Envoy Gateway), to reduce footprint, speed up onboarding, and enable non-Kubernetes environments.
- We deploy only one gateway instance in most setups.
- Envoy Gateway brings CRDs, controllers, and a multi-component lifecycle that:
- increases install time and RBAC surface,
- complicates upgrades and troubleshooting,
- couples us to Kubernetes even when users want a lighter path.
- Users without K8s controllers (or not on K8s at all) still want AIBrix features (routing, rate limiting by tokens/sec, prefix-cache awareness, batch API paths, etc.).
### Use Case
- for non-kubernetes users
- for users need simplification
### Proposed Solution
- Envoy runs as a single process with static or minimally-templated config (no Gateway API, no controllers).
- Generate Envoy bootstrap/listener/cluster config from AIBrix control-plane templates (Helm/JSON/YAML or aibrixctl).
- Hot-reload via SDS/xDS optional (envoy bootstrap points at our control-plane for dynamic clusters if needed).
Contributor guide
Research direction
Start by tracing the current Envoy Gateway deployment and AIBrix control-plane templates, then compare the Helm/JSON/YAML and aibrixctl paths mentioned in the proposal. Clarify the scope for static Envoy bootstrap, listener, and cluster configuration before implementation. Done should be an agreed, tested deployment path that works without Envoy Gateway and has a defined approach for optional hot reload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, json, kubernetes, yaml
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100