microsoft / microsoft/mssql-rs
Retain a rollback tag for pre-built Kerberos test images instead of only overwriting `latest`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 53
- Forks
- 14
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 137
Description
Problem statement
The Kerberos test images published by .pipeline/sync-container-images.yml are tagged latest and nothing else. The weekly schedule (cron: "0 3 * * 1", always: true) rebuilds and force-pushes those same tags every Monday, so the previous known-good image is destroyed on every run.
Affected tags (11 total):
kerberos/samba-dc:latest
kerberos/mssql-ad:latest
kerberos/client:{alpine318,alpine319,alpine320,alpine321,debian,ubuntu22,ubuntu24,rhel9,oracle9}-latest
Both PR and CI Kerberos stages consume these via .pipeline/templates/kerberos-test-template.yml, which defaults imageTag to latest and pulls only — there is no local build fallback, by design. That makes the images a hard dependency of the Kerberos stage.
The failure mode this creates: if a weekly run publishes a broken-but-pushable image (bad base image, upstream package regression, partially-applied change), latest is already overwritten by the time anyone notices. There is nothing to roll back to. Every Kerberos run on every PR fails until someone re-runs the sync pipeline from a known-good commit and waits for a full rebuild.
Note this is specifically about a bad image, not a missing one. A missing image fails at pull time and is obvious. A bad image fails later, during test execution, and looks like a product regression.
Proposed solution
Retain at least one prior generation so there is something to point at when latest is bad.
Sketch, not prescriptive:
- Before pushing a new
latest, retag the current one asprevious(orlatest-1). - Alternatively, push an immutable stamped tag alongside
lateston every run — date (2026-08-05), build ID, or source SHA — and keeplatestas a moving pointer. This generalises better than a singlepreviousslot and gives more than one generation of history. - Consumers keep defaulting to
latest; the stamped/previous tag is what you setimageTagto when you need to pin or roll back.
Rolling back then becomes a parameter change on the pipeline run rather than a rebuild.
Open questions
Should the consumer fall back automatically? I would lean no, and it should be a deliberate decision rather than a default. Automatically retrying a failed pull against previous gives a green pipeline running week-old images, which is the same "green but wrong" hazard that led us to reject a local-build fallback in #171. Explicit rollback (someone sets imageTag) keeps the failure visible while still making recovery fast. Worth confirming.
Retention. GHCR will accumulate stamped tags indefinitely. Needs either a retention policy or a deliberate decision to let them accumulate — these images are 1.5–3 GB each for the clients.
Client tag scheme. The clients currently encode the distro in the tag (kerberos/client:ubuntu22-latest) rather than the repository (kerberos/client-ubuntu22:latest). That works for latest, but it means any versioning scheme has to compose two dimensions into one tag string (ubuntu22-2026-08-05), and it makes per-distro digest pinning awkward. If we are changing the tagging scheme anyway, this is the moment to consider moving the distro into the repo name — it costs a republish either way, and the cost only grows.
Additional context
Context from #171 (consumer side) and #163 (producer side).
Relevant code:
.pipeline/sync-container-images.yml—BuildKerberosImagesjob builds and pushes the:latesttags to ACR;Mirror_To_GHCRcopies them to GHCR from the list in theimages.txtheredoc..pipeline/templates/kerberos-test-template.yml—imageRegistry/imageTagparameters; the pull step exits non-zero rather than falling back.
Worth noting the tags are currently static in practice — the weekly schedule only republishes from main, and the images were last built from the #163 branch run. This becomes an active risk the first Monday after #163's merge, when the scheduled job starts overwriting them on a cadence.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .pipeline/sync-container-images.yml, especially BuildKerberosImages, Mirror_To_GHCR, and the images.txt heredoc, then inspect .pipeline/templates/kerberos-test-template.yml. Trace how imageTag is passed and verify the chosen retention approach works for all listed images. Done means a prior image remains recoverable without changing the default consumer behavior, with retention and tag-format decisions resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100