NVIDIA / NVIDIA/nvcf

Epic: Move BDD fixture constants into chart defaults and remove masked configuration

Open
#1,868 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Summary

Audit the BDD suite for configuration that is hard-coded in fixtures or feature overlays even though it should be owned by a Helm chart (or derived from authoritative stack/chart configuration). Then remove the duplicated BDD inputs one chart at a time.

This epic is based on origin/main at 1309d5d64.

Goals

  • Make chart defaults authoritative for reusable component behavior.
  • Keep environment- and topology-specific choices explicit.
  • Stop BDD tests from masking broken value plumbing or stale defaults.
  • Do not hard-code released component versions in feature assertions; derive them from chart/stack configuration or verify artifact identity without a tag.

Recommended order

1. helm-nvcf-nats — config reloader image default
  • Move the complete upstream config-reloader image default into the NATS chart.
  • Decide whether the chart default should be updated from 0.23.0 to the stack/BDD value 0.24.0.
  • Keep the stack as an override-forwarding layer rather than the owner of the upstream default.
  • Remove the explicit image registry/repository/tag from the BDD scenario.
  • Derive any expected version from authoritative values, or assert image identity without hard-coding the tag.

Evidence:

2. helm-nvcf-api — Alpine/Kubernetes utility image default
  • Move the complete docker.io/alpine/k8s:1.37.0 default into the API chart.
  • Keep the stack able to override that image.
  • Remove the explicit image registry/repository/tag from the BDD scenario.
  • Derive any version assertion from authoritative values, or assert artifact identity without a tag.

Evidence:

3. helm-nvcf-llm-request-router — consolidate defaults and remove stale pins
  • Investigate whether the single-cluster fixture should omit its embedded load-balancer JSON and exercise Stargate's built-in permissive default.
  • If the narrowed algorithm set is test intent, document it as an intentional override instead of treating it as a default.
  • Remove multi-cluster fixture values that exactly repeat stack defaults: request-router replicas, grpcTls.mode, certificate secret name, PKI enablement, allowed domains, and DNS names.
  • Remove the stale OpenBao migrations image tag pin (0.19.1) and derive the expected value from chart/stack configuration (0.19.5 fallback today).
  • Remove explicit region-B image pull policies when chart fallbacks already provide IfNotPresent.
  • Remove feature overlays that repeat worker endpoint addresses and workload.kind: Deployment already supplied by the fixture/base.
  • Preserve the StatefulSet choice in the TLS scenario because it is behavioral test intent.

Evidence:

4. helm-nvca-operator — stop fixtures from masking value-plumbing defects
  • Preserve the chart's default validationPolicy.name: Unrestricted when merging the string-valued agentConfig.mergeConfig, then remove the fixture copy.
  • Fix the compute-plane stack key from ncaId to the chart-consumed ncaID; remove the fixture value that currently masks this mismatch.
  • Remove or correctly plumb selfManaged.identitySource: psat; no chart template currently consumes it.
  • Remove or relocate the unused top-level region; the chart reads selfManaged.region.
  • Remove the inert plural global.nodeSelectors.enabled; the compute stack uses singular global.nodeSelector.
  • Decide explicitly whether stargateQUICInsecure: false belongs in chart defaults before removing it from fixtures.
  • Keep canonical self-managed endpoint values explicit unless a separate design decision establishes safe chart defaults.

Evidence:

5. nvcf-gateway-routes — derive repeated listener/TLS configuration
  • Remove multi-cluster fixture values that exactly repeat base defaults for Gateway API enablement and listener names (nats, worker-tcp, llm-grpc, and llm-quic).
  • Investigate deriving grpcTls.dnsNames from global.workerEndpoints.llmRequestRouterAddress so the same authority is not specified twice.
  • Keep gateway names/namespaces, route enablement, and external reachability choices explicit because they are topology-specific.

Evidence:

6. helm-nvcf-llm-api-gateway — remove fixture copies of stack defaults
  • Remove grpcInsecure: true from the single-cluster fixture when the stack default supplies it.
  • Remove the explicit disabled ServiceMonitor setting when it is already derived from the disabled observability profile.
  • Keep replicaCount: 1 explicit because it is a local schedulability override.

Evidence:

7. External envoy-gateway-helm dependency — remove released-version literals
  • Replace the hard-coded v1.1.3 in the single-cluster EKS feature with a value derived from authoritative stack/chart configuration.
  • Do the same in the multi-cluster EKS feature.
  • Prefer artifact identity checks without a tag where the exact version is not the behavior under test.

Evidence:

8. Shared BDD fixtures and overlays — mechanical deduplication
  • Remove exact fixture copies of base defaults such as global.helm.sources.registry, global.image.registry, global.nodeSelectors.enabled, Gateway API enablement, and the NATS listener name.
  • Remove repeated observability.profile: disabled overlays when the selected fixture already disables observability.
  • Add focused tests where removing a fixture value reveals that a chart or stack fails to provide the intended default.
  • Keep this cleanup in small chart-scoped changes so failures identify the owning chart.

Keep explicit

The following values encode test-environment or topology intent and should not be defaulted merely to reduce fixture size:

  • Local storage classes and resource reductions.
  • Cassandra single-replica/resources/JVM startup tuning.
  • OpenBao injector replica count and issuer discovery.
  • NVCA feature-gate disables used to simulate local GPU density.
  • EKS endpoints, domains, and gateway resource references.
  • Route enablement needed for external reachability.
  • Image repositories and pull secrets tied to the test registry.
  • LLM/API gateway and vanity replica counts reduced for laptop schedulability.
  • Observability profile selection and opt-in OpenTelemetry collector behavior.

Test requirements for items 3–8

Follow the layered regression pattern from #1862 and Stephanie Baum's follow-up #1865. A successful Helm render, a healthy pod, or a fake-runner wiring test alone does not prove that the changed configuration reaches the product boundary.

  • Add a focused chart contract test that renders with the candidate value omitted and asserts the resolved default semantically. Parse embedded YAML/JSON with yq or an equivalent tool when configuration is stored inside a ConfigMap rather than relying only on text matching.
  • Cover explicit override behavior in the same test. Include partial, empty, or disabled cases when the chart supports them, so moving ownership does not break override precedence.
  • Add or extend a stack forwarding test proving Helmfile omits chart-owned defaults and forwards only explicitly configured, non-empty override keys.
  • Wire each focused test into the chart's normal make test target or an explicit premerge workflow step; adding an uninvoked script is insufficient.
  • Keep short BDD wiring coverage. It must prove the scenario does not re-inject the removed default and that the intended operator commands and observable assertions remain connected.
  • Keep released versions out of feature assertions and fake-runner contracts. Read expected versions from authoritative chart/stack values or assert artifact identity without the tag; arbitrary fixture-tag values are acceptable in wiring fixtures.
  • Exercise the affected path through live BDD when the regression can only be observed after rendering or installation. Cover both single- and multi-cluster topologies when the path or ownership differs between them.
  • When live BDD cannot run in premerge or locally, document the exact test entry point, prerequisites, and expected observable for QA. Do not present go test -short fake-runner coverage as live product verification.

Item-specific expectations:

  • Items 3–5: Begin with a rendered-values comparison, then test the chart/default merge and the relevant installed TLS, PKI, routing, or operator configuration—not merely pod readiness.
  • Item 6: Add or extend an LLM API gateway render contract for the omitted defaults and preserve an installed/rendered BDD observable.
  • Item 7: Derive the Envoy Gateway version from authoritative configuration in short wiring coverage and keep the feature assertion version-independent.
  • Item 8: For every mechanical removal, compare the effective rendered output before and after; retain the existing live scenario that exercises the affected chart.

Acceptance criteria

  • Each completed item identifies one authoritative default owner: chart first, with stack/environment layers only overriding it.
  • BDD passes without supplying values that merely reproduce that owner.
  • Released component versions are not duplicated in feature assertions.
  • Topology-specific and resource-constrained test values remain explicit.
  • Any newly discovered inert or misspelled value is either fixed or removed with a regression test.
  • Documentation/comments that describe obsolete default propagation are updated.

Execution note

Work this epic one checked section at a time. The first two sections are high-confidence, narrow changes. Sections 3–5 include ownership/design decisions and should begin with a rendered-values comparison before changing behavior.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the unchecked items in tests/bdd/fixtures/self-managed-local-bdd.yaml and self-managed-local-bdd-multi.yaml, comparing them with the referenced chart values and stack files. Read the regression approach from #1862 and #1865, then run the affected chart, stack-forwarding, and BDD checks. Done means chart-owned defaults are no longer duplicated, overrides remain covered, and released versions are not hard-coded in assertions.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, yaml
Domain
devops, infrastructure, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.