Pull CLI Docker images matching the version of the CLI
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 19h 46m
- Merged PRs (30d)
- 141
Description
### User Story
As a user of the Sleeper Docker tools CLI, I want the `sleeper` command and the Docker images it runs to come
from the same version of Sleeper, so that a change to the CLI script can rely on what's inside the images.
### Description / Background
Follow-up from #7903, where this was raised in review.
The CLI script and the images it uses are versioned independently, and nothing keeps them aligned:
* `scripts/cli/runInDocker.sh` has no version of its own. The install script and `sleeper cli upgrade` fetch it
from the `develop` branch, or from a local checkout when installed with `--useLocalRepo`.
* Images are always pulled with the tag `latest` (`pull_docker_images` in `scripts/cli/runInDocker.sh`), because
that's the only tag the CLI pipeline publishes (`.github/scripts/planCliBuild.sh`). For `ghcr.io/gchq` that tag is
rebuilt from `develop` once a day.
* `sleeper version` reports the version baked into the `sleeper-local` image (`/sleeper/version.txt`, written by
`scripts/cli/environment/buildMaven.sh`), so it describes the images, not the script.
So the script can be ahead of or behind the images in several ordinary situations: between a merge to `develop` and
the next nightly image build; when installed with `--useLocalRepo` from a checkout that isn't current `develop`; and
for anyone whose registry mirrors images on a different schedule.
#7903 added `--useLocalVersion`, which pulls the tag named by the checkout's `java/pom.xml` instead of `latest`.
That only partly helps: it needs a registry that publishes version tags, which ours doesn't, and a version such as
`0.37.6-SNAPSHOT` covers a whole release line rather than a single commit.
What we'd like is for the CLI to pull the images belonging to the version of the CLI you're running, from the
default registry, without any extra options.
### Acceptance Criteria
**Given** the CLI pipeline has run for version 0.37.6-SNAPSHOT
**When** I install the CLI with the documented install command
**Then** the images pulled are tagged with that version rather than `latest`
**Given** I have the CLI installed at some version
**When** I run `sleeper cli upgrade`
**Then** the command and the images are both updated, and the images match the version of the updated command
**Given** I installed with `--useLocalRepo`
**When** images are pulled
**Then** the tag used is the version in that checkout's `java/pom.xml`
**Given** a custom registry that publishes only `latest`
**When** I install or pull images
**Then** I get a clear message about the missing version tag rather than an unexplained pull failure
### Technical Notes / Implementation Details
Likely pieces of work:
1. Publish version tags from the CLI pipeline. `.github/scripts/planCliBuild.sh` currently emits only
`:latest`; it needs a version tag as well, read from `java/pom.xml`, with
`.github/workflows/docker-cli-image.yaml` pushing both.
2. Stop the version tags being deleted. The clean-up step in `docker-cli-image.yaml` runs `DeleteGHCRVersions` with
`ignoreTagsPattern=^latest$` and `keepMostRecent=4`, so released versions would be deleted after a few nightly
builds. The pattern needs to cover version tags, and we should decide what retention we want.
3. Give the script a version of its own to pull by. Options: a constant in `runInDocker.sh` maintained by
`scripts/dev/updateVersionNumber.sh` alongside the poms, `pyproject.toml` and Cargo files; or a version file
installed next to the command. When installed with `--useLocalRepo`, the version should still come from the
checkout's `java/pom.xml`, which is what `get_version_tag` already does.
4. Decide the behaviour when the version tag isn't in the registry — fall back to `latest` with a warning, or fail
with a message naming the missing tag. This matters for custom registries and for the window before a new
version's images have been published.
5. Fold `--useLocalVersion` into this. Once the script pulls its own version by default, the option is redundant and
can be removed, leaving `--useLocalRepo` for tying an install to a checkout.
6. Update `docs/development/publishing.md`, `docs/developer-guide.md` and `docs/development/release-process.md` for
whatever we settle on, including what a custom publishing pipeline now has to publish.
Out of scope: making images match a specific commit. A version identifies a release line, so the only images that
match a checkout exactly are ones built locally with `scripts/cli/buildAll.sh`. Worth stating in the docs rather
than solving here.
### Dependencies / Blockers
Depends on #7903.
Contributor guide
Research direction
Start with scripts/cli/runInDocker.sh, .github/scripts/planCliBuild.sh, .github/workflows/docker-cli-image.yaml, and scripts/dev/updateVersionNumber.sh; trace how versions and image tags are currently obtained and published. Review the three listed documentation files and #7903 before choosing the missing-tag and retention behavior. Done means install, upgrade, local-repository, and custom-registry scenarios meet the acceptance criteria with matching documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, java, shell
- Domain
- ci-cd, cli, documentation, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100