nebari-dev / nebari-dev/llm-serving-pack
Helm chart install validation in CI (not just lint)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3
- Forks
- 3
- Avg merge
- 16h 31m
- Merged PRs (30d)
- 11
Description
Problem
The current Lint Helm chart CI job runs helm lint and helm template, which only catches structural problems with the chart. It does not catch:
- Templates that render valid YAML but fail to install on a real cluster (CRD ordering, RBAC issues, missing CRDs, namespace assumptions).
- Operator startup crashes once the rendered manifests are actually applied.
- Webhook bootstrap problems (cert injection, conversion, etc.).
- Helm hook failures.
Issue #59 (collapse the pack into a single namespace) was caught at code review but the actual install would have failed in several ways the lint job missed. The chart having a separate namespace.yaml template was discovered only because a human read the diff.
Proposal
Add a CI job that:
- Spins up a
kind(ork3d) cluster in the GitHub Actions runner. - Installs the prerequisites the chart documents (cert-manager, gateway-api CRDs, Envoy Gateway + AI Gateway, KServe gateway-api-inference-extension CRDs).
- Runs
helm install nebari-llm-serving ./charts/nebari-llm-serving --create-namespace -n nebari-llm-serving-system. - Waits for operator and key-manager pods to become Ready.
- Applies a sample
LLMModelCR (the one inexamples/or a CI-only minimal one). - Asserts the operator reconciles it without error events and the expected SecurityPolicies / HTTPRoutes get created.
- Tears the cluster down.
This does not exercise actual model serving (no GPUs in the runner), but it catches every "the chart broke at install time" class of bug.
Notes
- The job will be slower than lint - probably 5-8 minutes. Worth it.
- Cache the prerequisite chart pulls between runs.
- If kind setup is too expensive for every PR, gate this on
pathsfilters touchingcharts/,operator/config/, orexamples/.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing Lint Helm chart GitHub Actions job and the chart under charts/nebari-llm-serving; review the prerequisite versions and install commands documented by the project. Use the sample LLMModel in examples/ or a minimal CI fixture, and consider the paths operator/config/ and examples/ for CI gating. Done means a kind or k3d job installs the chart, waits for the operator and key-manager, verifies reconciliation plus SecurityPolicies and HTTPRoutes, and tears the cluster down.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go, helm, kubernetes
- Domain
- ci-cd, devops, infrastructure, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100