nebari-dev / nebari-dev/llm-serving-pack
Implement envoyAIGateway.install flag in Helm chart
@marcelovilla is already working on this.
Since Aug 27, 2026.
- Dominant language
- Go
- Stars
- 3
- Forks
- 3
- Avg merge
- 16h 31m
- Merged PRs (30d)
- 11
Description
Problem
The chart's values.yaml has a placeholder for installing the Envoy AI Gateway:
envoyAIGateway:
# install: if true, installs the Envoy AI Gateway (not yet implemented)
install: false
But no templates actually do anything when this is set to true. The AI Gateway is required for the pack's core functionality (token counting, rate limiting, API key enforcement via SecurityPolicies, AIGatewayRoutes) and the operator generates AIGatewayRoute and SecurityPolicy resources that depend on the AI Gateway CRDs being installed on the cluster.
Today users must install the AI Gateway manually as separate ArgoCD apps or Helm releases.
Envoy AI Gateway details (upstream)
Based on https://aigateway.envoyproxy.io/docs/getting-started/installation/ and https://github.com/envoyproxy/ai-gateway:
- OCI registry:
oci://docker.io/envoyproxy - Two charts:
ai-gateway-crds-helm(CRDs:AIGatewayRoute,AIServiceBackend,BackendSecurityPolicy,GatewayConfig,MCPRoute)ai-gateway-helm(controller + webhooks)
- Current stable version:
v0.5.0(as of January 2026) - Install namespace:
envoy-ai-gateway-system - Prerequisites:
- Kubernetes 1.32+
- Envoy Gateway v1.6+ with the AI Gateway-compatible values applied (enables Extension Manager pointing at
aieg.envoy-ai-gateway-system.svc.cluster.local:1063)
Proposed implementation
Add an optional sub-chart (or Helm template) that installs the AI Gateway when envoyAIGateway.install: true. Because there are two charts, the pack would need to install both.
Option A: Helm sub-chart dependencies
Add two dependencies in charts/nebari-llm-serving/Chart.yaml, both conditional on envoyAIGateway.install:
dependencies:
- name: ai-gateway-crds-helm
version: \"0.5.0\"
repository: \"oci://docker.io/envoyproxy\"
condition: envoyAIGateway.install
- name: ai-gateway-helm
version: \"0.5.0\"
repository: \"oci://docker.io/envoyproxy\"
condition: envoyAIGateway.install
Pros: reuses the upstream charts, gets updates for free.
Cons: the pack has less control over install ordering/timing. In ArgoCD, CRDs must install before the controller. Helm normally handles ordering but cross-chart dependencies in sub-charts can be finicky.
Option B: Ship two templated ArgoCD Applications
Under charts/nebari-llm-serving/templates/envoy-ai-gateway/, render two Application resources (one for CRDs, one for the controller) with sync-waves that enforce ordering. This is the pattern the pack already uses for other foundational resources.
Recommendation: Option B. The pack is fundamentally ArgoCD-driven and already ships ArgoCD Applications as templates. This gives explicit control over sync ordering (CRDs first) and fits the rest of the chart's conventions. Sub-chart dependencies would fight the ArgoCD lifecycle.
Values to expose
envoyAIGateway:
install: false
version: v0.5.0 # pinned AI Gateway version (both charts use same tag)
namespace: envoy-ai-gateway-system
# Optional override of the OCI registry for air-gapped environments
registry: oci://docker.io/envoyproxy
Envoy Gateway compatibility
The AI Gateway requires Envoy Gateway to be configured with the Extension Manager enabled, pointing at the AI Gateway controller service. Upstream provides a values file:
https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/manifests/envoy-gateway-values.yaml
nebari-infrastructure-core installs Envoy Gateway as a foundational service but may not use these values by default. We may need to either:
- Document that users must apply these values to their Envoy Gateway install, or
- File an issue against nebari-infrastructure-core to make the Envoy Gateway values configurable/aware of the AI Gateway extension, or
- Install our own Envoy Gateway on top of the nic one (not recommended)
Option 2 is cleanest long-term. Option 1 is fine for the initial implementation.
Acceptance criteria
- Setting
envoyAIGateway.install: truein Helm values installs both the CRDs chart and the controller chart, in the correct order (CRDs first). - The operator's existing
AIGatewayRouteandSecurityPolicygeneration works against the installed CRDs without further config. - When
install: false, nothing extra is deployed and the chart installs normally (assumes the user has installed the AI Gateway separately or via another chart/ArgoCD app). - Documented in the README with a version compatibility note and the Envoy Gateway extension manager caveat.
- Working example in
examples/envoy-ai-gateway.yaml(shipped today as a workaround, should continue to work after the feature lands for users who prefer manual installation).
Workaround until implemented
See examples/envoy-ai-gateway.yaml for a two-Application ArgoCD manifest that installs both charts. Or via Helm directly:
# 1. CRDs first
helm upgrade -i aieg-crd oci://docker.io/envoyproxy/ai-gateway-crds-helm \\
--version v0.5.0 \\
--namespace envoy-ai-gateway-system \\
--create-namespace
# 2. Envoy Gateway with AI-Gateway-compatible values (if not already done)
helm upgrade -i eg oci://docker.io/envoyproxy/gateway-helm \\
--version v1.6.0 \\
--namespace envoy-gateway-system \\
--create-namespace \\
-f https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/manifests/envoy-gateway-values.yaml
# 3. Controller
helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm \\
--version v0.5.0 \\
--namespace envoy-ai-gateway-system
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.