Epic: Move BDD fixture constants into chart defaults and remove masked configuration
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Summary
Audit the BDD suite for configuration that is hard-coded in fixtures or feature overlays even though it should be owned by a Helm chart (or derived from authoritative stack/chart configuration). Then remove the duplicated BDD inputs one chart at a time.
This epic is based on origin/main at 1309d5d64.
Goals
- Make chart defaults authoritative for reusable component behavior.
- Keep environment- and topology-specific choices explicit.
- Stop BDD tests from masking broken value plumbing or stale defaults.
- Do not hard-code released component versions in feature assertions; derive them from chart/stack configuration or verify artifact identity without a tag.
Recommended order
1. helm-nvcf-nats — config reloader image default
- Move the complete upstream config-reloader image default into the NATS chart.
- Decide whether the chart default should be updated from
0.23.0to the stack/BDD value0.24.0. - Keep the stack as an override-forwarding layer rather than the owner of the upstream default.
- Remove the explicit image registry/repository/tag from the BDD scenario.
- Derive any expected version from authoritative values, or assert image identity without hard-coding the tag.
Evidence:
- BDD supplies the complete image.
- The self-managed stack already supplies the complete default.
- The underlying chart leaves registry/repository empty and has a different tag.
2. helm-nvcf-api — Alpine/Kubernetes utility image default
- Move the complete
docker.io/alpine/k8s:1.37.0default into the API chart. - Keep the stack able to override that image.
- Remove the explicit image registry/repository/tag from the BDD scenario.
- Derive any version assertion from authoritative values, or assert artifact identity without a tag.
Evidence:
- BDD supplies the complete image.
- The self-managed stack contains the exact default.
- The API chart currently leaves the image fields empty.
3. helm-nvcf-llm-request-router — consolidate defaults and remove stale pins
- Investigate whether the single-cluster fixture should omit its embedded load-balancer JSON and exercise Stargate's built-in permissive default.
- If the narrowed algorithm set is test intent, document it as an intentional override instead of treating it as a default.
- Remove multi-cluster fixture values that exactly repeat stack defaults: request-router replicas,
grpcTls.mode, certificate secret name, PKI enablement, allowed domains, and DNS names. - Remove the stale OpenBao migrations image tag pin (
0.19.1) and derive the expected value from chart/stack configuration (0.19.5fallback today). - Remove explicit region-B image pull policies when chart fallbacks already provide
IfNotPresent. - Remove feature overlays that repeat worker endpoint addresses and
workload.kind: Deploymentalready supplied by the fixture/base. - Preserve the StatefulSet choice in the TLS scenario because it is behavioral test intent.
Evidence:
- Single-cluster fixture embeds load-balancer JSON.
- Chart documentation describes the built-in permissive default.
- Multi-cluster fixture repeats request-router/TLS/PKI settings.
- The self-managed base owns those settings.
- PKI feature pins the migrations version.
- Current stack fallback for the migrations image.
- Region-B helper explicitly sets pull policies.
4. helm-nvca-operator — stop fixtures from masking value-plumbing defects
- Preserve the chart's default
validationPolicy.name: Unrestrictedwhen merging the string-valuedagentConfig.mergeConfig, then remove the fixture copy. - Fix the compute-plane stack key from
ncaIdto the chart-consumedncaID; remove the fixture value that currently masks this mismatch. - Remove or correctly plumb
selfManaged.identitySource: psat; no chart template currently consumes it. - Remove or relocate the unused top-level
region; the chart readsselfManaged.region. - Remove the inert plural
global.nodeSelectors.enabled; the compute stack uses singularglobal.nodeSelector. - Decide explicitly whether
stargateQUICInsecure: falsebelongs in chart defaults before removing it from fixtures. - Keep canonical self-managed endpoint values explicit unless a separate design decision establishes safe chart defaults.
Evidence:
- Single-cluster compute fixture restates operator configuration.
- Static-registration fixture uses the chart-consumed ncaID.
- Compute-plane stack currently emits ncaId.
- Operator chart declares ncaID.
- Operator chart default validation policy.
- Template reads selfManaged.region.
5. nvcf-gateway-routes — derive repeated listener/TLS configuration
- Remove multi-cluster fixture values that exactly repeat base defaults for Gateway API enablement and listener names (
nats,worker-tcp,llm-grpc, andllm-quic). - Investigate deriving
grpcTls.dnsNamesfromglobal.workerEndpoints.llmRequestRouterAddressso the same authority is not specified twice. - Keep gateway names/namespaces, route enablement, and external reachability choices explicit because they are topology-specific.
Evidence:
- Multi-cluster fixture repeats listener configuration.
- Self-managed base already owns the Gateway API/listener defaults.
6. helm-nvcf-llm-api-gateway — remove fixture copies of stack defaults
- Remove
grpcInsecure: truefrom the single-cluster fixture when the stack default supplies it. - Remove the explicit disabled ServiceMonitor setting when it is already derived from the disabled observability profile.
- Keep
replicaCount: 1explicit because it is a local schedulability override.
Evidence:
7. External envoy-gateway-helm dependency — remove released-version literals
- Replace the hard-coded
v1.1.3in the single-cluster EKS feature with a value derived from authoritative stack/chart configuration. - Do the same in the multi-cluster EKS feature.
- Prefer artifact identity checks without a tag where the exact version is not the behavior under test.
Evidence:
8. Shared BDD fixtures and overlays — mechanical deduplication
- Remove exact fixture copies of base defaults such as
global.helm.sources.registry,global.image.registry,global.nodeSelectors.enabled, Gateway API enablement, and the NATS listener name. - Remove repeated
observability.profile: disabledoverlays when the selected fixture already disables observability. - Add focused tests where removing a fixture value reveals that a chart or stack fails to provide the intended default.
- Keep this cleanup in small chart-scoped changes so failures identify the owning chart.
Keep explicit
The following values encode test-environment or topology intent and should not be defaulted merely to reduce fixture size:
- Local storage classes and resource reductions.
- Cassandra single-replica/resources/JVM startup tuning.
- OpenBao injector replica count and issuer discovery.
- NVCA feature-gate disables used to simulate local GPU density.
- EKS endpoints, domains, and gateway resource references.
- Route enablement needed for external reachability.
- Image repositories and pull secrets tied to the test registry.
- LLM/API gateway and vanity replica counts reduced for laptop schedulability.
- Observability profile selection and opt-in OpenTelemetry collector behavior.
Test requirements for items 3–8
Follow the layered regression pattern from #1862 and Stephanie Baum's follow-up #1865. A successful Helm render, a healthy pod, or a fake-runner wiring test alone does not prove that the changed configuration reaches the product boundary.
- Add a focused chart contract test that renders with the candidate value omitted and asserts the resolved default semantically. Parse embedded YAML/JSON with
yqor an equivalent tool when configuration is stored inside a ConfigMap rather than relying only on text matching. - Cover explicit override behavior in the same test. Include partial, empty, or disabled cases when the chart supports them, so moving ownership does not break override precedence.
- Add or extend a stack forwarding test proving Helmfile omits chart-owned defaults and forwards only explicitly configured, non-empty override keys.
- Wire each focused test into the chart's normal
make testtarget or an explicit premerge workflow step; adding an uninvoked script is insufficient. - Keep short BDD wiring coverage. It must prove the scenario does not re-inject the removed default and that the intended operator commands and observable assertions remain connected.
- Keep released versions out of feature assertions and fake-runner contracts. Read expected versions from authoritative chart/stack values or assert artifact identity without the tag; arbitrary
fixture-tagvalues are acceptable in wiring fixtures. - Exercise the affected path through live BDD when the regression can only be observed after rendering or installation. Cover both single- and multi-cluster topologies when the path or ownership differs between them.
- When live BDD cannot run in premerge or locally, document the exact test entry point, prerequisites, and expected observable for QA. Do not present
go test -shortfake-runner coverage as live product verification.
Item-specific expectations:
- Items 3–5: Begin with a rendered-values comparison, then test the chart/default merge and the relevant installed TLS, PKI, routing, or operator configuration—not merely pod readiness.
- Item 6: Add or extend an LLM API gateway render contract for the omitted defaults and preserve an installed/rendered BDD observable.
- Item 7: Derive the Envoy Gateway version from authoritative configuration in short wiring coverage and keep the feature assertion version-independent.
- Item 8: For every mechanical removal, compare the effective rendered output before and after; retain the existing live scenario that exercises the affected chart.
Acceptance criteria
- Each completed item identifies one authoritative default owner: chart first, with stack/environment layers only overriding it.
- BDD passes without supplying values that merely reproduce that owner.
- Released component versions are not duplicated in feature assertions.
- Topology-specific and resource-constrained test values remain explicit.
- Any newly discovered inert or misspelled value is either fixed or removed with a regression test.
- Documentation/comments that describe obsolete default propagation are updated.
Execution note
Work this epic one checked section at a time. The first two sections are high-confidence, narrow changes. Sections 3–5 include ownership/design decisions and should begin with a rendered-values comparison before changing behavior.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the unchecked items in tests/bdd/fixtures/self-managed-local-bdd.yaml and self-managed-local-bdd-multi.yaml, comparing them with the referenced chart values and stack files. Read the regression approach from #1862 and #1865, then run the affected chart, stack-forwarding, and BDD checks. Done means chart-owned defaults are no longer duplicated, overrides remain covered, and released versions are not hard-coded in assertions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes, yaml
- Domain
- devops, infrastructure, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100