linux_new: Template regression tests and published-package test mode
- Dominant language
- Jinja
- Stars
- 168
- Forks
- 77
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 12
Description
Part of #1445
---
## Background
### No template-layer tests
The pipeline has no fast, offline tests for the template layer. The existing TestContainers-
based tests (`AptOperationsTest`, `ApkOperationsTest`, `DnfOperationsTest`) validate package
installation inside real distro containers, but only run against locally-built packages after
a full build completes. There is currently nothing that catches a silent change to template
logic before a build is attempted.
This creates a specific risk for the template de-duplication work in child issues #1 and #2:
shared Jinja2 templates that use version conditionals could produce subtly wrong output for
one version without being detected unless that specific version's full build is run and
inspected manually.
### No way to test against already-published packages
The existing TestContainers tests require a local package build to run. This means changes
to the pipeline scripts, `build.sh`, or Dockerfiles cannot be validated against a known-good
binary without triggering a full package build first. A `TEST_AGAINST_PUBLISHED` mode would
allow the installation tests to pull a previously published package from Artifactory and
validate that the test harness itself works correctly before any new packages are built.
---
## Goal
1. Add a lightweight offline regression test suite that renders all templates against fixed
synthetic inputs and diffs the output against committed golden files. This is the safety
net for the de-duplication work in child issues #1 and #2.
2. Add a `TEST_AGAINST_PUBLISHED` mode to the existing TestContainers tests so that pipeline
script and harness changes can be validated against a previously published package without
requiring a local build.
3. Wire both into the Jenkinsfile as a `Test Templates` stage that runs early, before
`Generate Spec File`.
---
## Critical ordering requirement
> ⚠️ **Step 1 of this issue — generating and committing the golden files — must be completed
> before any template files are changed in child issues #1 or #2.**
>
> The golden files capture the output of the _current_ templates as the correctness baseline.
> If template changes land first, the baseline is lost and equivalence can no longer be
> verified automatically.
>
> **Commit the golden files as a standalone commit/PR with no other changes.**
---
## Scope
### Part A — Template regression test suite
**Directory structure to create:**
Contributor guide
Assessment
This issue has not been assessed yet.