gchq / gchq/sleeper

Pull CLI Docker images matching the version of the CLI

Open
#8,140 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

### User Story

As a user of the Sleeper Docker tools CLI, I want the `sleeper` command and the Docker images it runs to come
from the same version of Sleeper, so that a change to the CLI script can rely on what's inside the images.

### Description / Background

Follow-up from #7903, where this was raised in review.

The CLI script and the images it uses are versioned independently, and nothing keeps them aligned:

* `scripts/cli/runInDocker.sh` has no version of its own. The install script and `sleeper cli upgrade` fetch it
from the `develop` branch, or from a local checkout when installed with `--useLocalRepo`.
* Images are always pulled with the tag `latest` (`pull_docker_images` in `scripts/cli/runInDocker.sh`), because
that's the only tag the CLI pipeline publishes (`.github/scripts/planCliBuild.sh`). For `ghcr.io/gchq` that tag is
rebuilt from `develop` once a day.
* `sleeper version` reports the version baked into the `sleeper-local` image (`/sleeper/version.txt`, written by
`scripts/cli/environment/buildMaven.sh`), so it describes the images, not the script.

So the script can be ahead of or behind the images in several ordinary situations: between a merge to `develop` and
the next nightly image build; when installed with `--useLocalRepo` from a checkout that isn't current `develop`; and
for anyone whose registry mirrors images on a different schedule.

#7903 added `--useLocalVersion`, which pulls the tag named by the checkout's `java/pom.xml` instead of `latest`.
That only partly helps: it needs a registry that publishes version tags, which ours doesn't, and a version such as
`0.37.6-SNAPSHOT` covers a whole release line rather than a single commit.

What we'd like is for the CLI to pull the images belonging to the version of the CLI you're running, from the
default registry, without any extra options.

### Acceptance Criteria

**Given** the CLI pipeline has run for version 0.37.6-SNAPSHOT
**When** I install the CLI with the documented install command
**Then** the images pulled are tagged with that version rather than `latest`

**Given** I have the CLI installed at some version
**When** I run `sleeper cli upgrade`
**Then** the command and the images are both updated, and the images match the version of the updated command

**Given** I installed with `--useLocalRepo`
**When** images are pulled
**Then** the tag used is the version in that checkout's `java/pom.xml`

**Given** a custom registry that publishes only `latest`
**When** I install or pull images
**Then** I get a clear message about the missing version tag rather than an unexplained pull failure

### Technical Notes / Implementation Details

Likely pieces of work:

1. Publish version tags from the CLI pipeline. `.github/scripts/planCliBuild.sh` currently emits only
`:latest`; it needs a version tag as well, read from `java/pom.xml`, with
`.github/workflows/docker-cli-image.yaml` pushing both.
2. Stop the version tags being deleted. The clean-up step in `docker-cli-image.yaml` runs `DeleteGHCRVersions` with
`ignoreTagsPattern=^latest$` and `keepMostRecent=4`, so released versions would be deleted after a few nightly
builds. The pattern needs to cover version tags, and we should decide what retention we want.
3. Give the script a version of its own to pull by. Options: a constant in `runInDocker.sh` maintained by
`scripts/dev/updateVersionNumber.sh` alongside the poms, `pyproject.toml` and Cargo files; or a version file
installed next to the command. When installed with `--useLocalRepo`, the version should still come from the
checkout's `java/pom.xml`, which is what `get_version_tag` already does.
4. Decide the behaviour when the version tag isn't in the registry — fall back to `latest` with a warning, or fail
with a message naming the missing tag. This matters for custom registries and for the window before a new
version's images have been published.
5. Fold `--useLocalVersion` into this. Once the script pulls its own version by default, the option is redundant and
can be removed, leaving `--useLocalRepo` for tying an install to a checkout.
6. Update `docs/development/publishing.md`, `docs/developer-guide.md` and `docs/development/release-process.md` for
whatever we settle on, including what a custom publishing pipeline now has to publish.

Out of scope: making images match a specific commit. A version identifies a release line, so the only images that
match a checkout exactly are ones built locally with `scripts/cli/buildAll.sh`. Worth stating in the docs rather
than solving here.

### Dependencies / Blockers

Depends on #7903.

Contributor guide

Open the contributing guide

Research direction

Start with scripts/cli/runInDocker.sh, .github/scripts/planCliBuild.sh, .github/workflows/docker-cli-image.yaml, and scripts/dev/updateVersionNumber.sh; trace how versions and image tags are currently obtained and published. Review the three listed documentation files and #7903 before choosing the missing-tag and retention behavior. Done means install, upgrade, local-repository, and custom-registry scenarios meet the acceptance criteria with matching documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, java, shell
Domain
ci-cd, cli, documentation, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.