adoptium / adoptium/installer

linux_new: Template regression tests and published-package test mode

Open
#1,449 0 comments 0 reactions 1 assignee Claimed by @steelhead31 View on GitHub
enhancement Linux
Dominant language
Jinja
Stars
168
Forks
77
Avg merge
2d 14h
Merged PRs (30d)
12

Description

Part of #1445
---

## Background

### No template-layer tests

The pipeline has no fast, offline tests for the template layer. The existing TestContainers-
based tests (`AptOperationsTest`, `ApkOperationsTest`, `DnfOperationsTest`) validate package
installation inside real distro containers, but only run against locally-built packages after
a full build completes. There is currently nothing that catches a silent change to template
logic before a build is attempted.

This creates a specific risk for the template de-duplication work in child issues #1 and #2:
shared Jinja2 templates that use version conditionals could produce subtly wrong output for
one version without being detected unless that specific version's full build is run and
inspected manually.

### No way to test against already-published packages

The existing TestContainers tests require a local package build to run. This means changes
to the pipeline scripts, `build.sh`, or Dockerfiles cannot be validated against a known-good
binary without triggering a full package build first. A `TEST_AGAINST_PUBLISHED` mode would
allow the installation tests to pull a previously published package from Artifactory and
validate that the test harness itself works correctly before any new packages are built.

---

## Goal

1. Add a lightweight offline regression test suite that renders all templates against fixed
synthetic inputs and diffs the output against committed golden files. This is the safety
net for the de-duplication work in child issues #1 and #2.
2. Add a `TEST_AGAINST_PUBLISHED` mode to the existing TestContainers tests so that pipeline
script and harness changes can be validated against a previously published package without
requiring a local build.
3. Wire both into the Jenkinsfile as a `Test Templates` stage that runs early, before
`Generate Spec File`.

---

## Critical ordering requirement

> ⚠️ **Step 1 of this issue — generating and committing the golden files — must be completed
> before any template files are changed in child issues #1 or #2.**
>
> The golden files capture the output of the _current_ templates as the correctness baseline.
> If template changes land first, the baseline is lost and equivalence can no longer be
> verified automatically.
>
> **Commit the golden files as a standalone commit/PR with no other changes.**

---

## Scope

### Part A — Template regression test suite

**Directory structure to create:**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.