vllm-project / vllm-project/aibrix

[Discussion] Simplify AIBrix deployment by removing Envoy Gateway

Open
#1,656 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🚀 Feature Description and Motivation

Today AIBrix recommends/ships Envoy Gateway, but we typically run a `single` gateway instance. Envoy Gateway adds installation and controller complexity that many users don’t need. Propose an option to deploy vanilla Envoy + ext-proc (External Processing), fully managed by AIBrix (no Envoy Gateway), to reduce footprint, speed up onboarding, and enable non-Kubernetes environments.

- We deploy only one gateway instance in most setups.
- Envoy Gateway brings CRDs, controllers, and a multi-component lifecycle that:
- increases install time and RBAC surface,
- complicates upgrades and troubleshooting,
- couples us to Kubernetes even when users want a lighter path.
- Users without K8s controllers (or not on K8s at all) still want AIBrix features (routing, rate limiting by tokens/sec, prefix-cache awareness, batch API paths, etc.).

### Use Case

- for non-kubernetes users
- for users need simplification

### Proposed Solution

- Envoy runs as a single process with static or minimally-templated config (no Gateway API, no controllers).
- Generate Envoy bootstrap/listener/cluster config from AIBrix control-plane templates (Helm/JSON/YAML or aibrixctl).
- Hot-reload via SDS/xDS optional (envoy bootstrap points at our control-plane for dynamic clusters if needed).

Contributor guide

Open the contributing guide

Research direction

Start by tracing the current Envoy Gateway deployment and AIBrix control-plane templates, then compare the Helm/JSON/YAML and aibrixctl paths mentioned in the proposal. Clarify the scope for static Envoy bootstrap, listener, and cluster configuration before implementation. Done should be an agreed, tested deployment path that works without Envoy Gateway and has a defined approach for optional hot reload.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, json, kubernetes, yaml
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.