nebari-dev / nebari-dev/llm-serving-pack
Move operator to aigateway.envoyproxy.io/v1beta1 and update examples to Envoy Gateway v1.8.3 / AI Gateway v1.1.0
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3
- Forks
- 3
- Avg merge
- 16h 31m
- Merged PRs (30d)
- 11
Description
Problem
The operator writes six objects as aigateway.envoyproxy.io/v1alpha1 (reconcilers/routing.go, reconcilers/passthrough.go). Envoy AI Gateway v0.6.0 promoted AIGatewayRoute, AIServiceBackend and BackendSecurityPolicy to v1beta1; v1.x serves v1alpha1 only as deprecated (deprecated: true, storage: false), so every write emits a deprecation warning and the version will be dropped in some later release.
examples/envoy-ai-gateway.yaml and examples/envoy-gateway.yaml still pin ai-gateway v0.5.0 and Envoy Gateway v1.6.2. ai-gateway v0.5.0 has a controller crash loop that freezes config for every service on a shared gateway (nil dereference in PostRouteModify when the InferencePool controller has not synced yet; fixed in https://github.com/envoyproxy/ai-gateway/pull/1951, first released in v0.6.0). We should not be recommending it.
Compatibility already verified
A customer is bumping to Envoy Gateway v1.8.3 and ai-gateway v1.1.0 without any pack change (https://github.com/Concepts-NREC/Nebari-GitOps/pull/87, https://github.com/Concepts-NREC/Nebari-GitOps/pull/88, https://github.com/Concepts-NREC/Nebari-GitOps/pull/89). This is safe because:
v1alpha1is still served by v1.1.0.- The two v0.6 breaking changes do not apply: the operator never sets
AIGatewayRoute.spec.filterConfig, andschema.versionis only emitted forPassthroughModel(asv1, not a path prefix). - The operator does not use
EnvoyPatchPolicyor an OIDCSecurityPolicy, the EG v1.7/v1.8 changes that affect extension users. - ai-gateway v1.1.0 reads InferencePool via GIE v1.0.2 client while we install GIE v1.5.0 CRDs; both are the stable
inference.networking.k8s.io/v1API.
Fix
- Change the six
apiVersionliterals toaigateway.envoyproxy.io/v1beta1and the RBAC marker if needed. Keep the field set identical; nothing we set changed shape. examples/: Envoy Gatewayv1.6.2→v1.8.3, ai-gateway CRDs and controllerv0.5.0→v1.1.0. AddextensionManager.failOpen: trueandcontroller.leaderElection.enabled: false(single replica) with a comment explaining the crash-loop window; see the customer PRs above for wording.- Document the tested version triple (Envoy Gateway / AI Gateway / GIE CRDs) in the README so the ordering across NIC and the pack is written down somewhere.
- Note in the changelog that the pack was verified on v0.5.0 and v1.1.0 so installs mid-upgrade are supported.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in reconcilers/routing.go and reconcilers/passthrough.go, locating the six apiVersion literals and the RBAC marker. Then update examples/envoy-ai-gateway.yaml and examples/envoy-gateway.yaml using the requested Envoy Gateway and AI Gateway versions and settings, and read the customer PRs for wording. Done means the README documents the tested version triple and the changelog records compatibility across the specified versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 64/100