Proposal ( / dream): how I'd like our versioning/publishing actions to behave

Below is a description of how I'd like our automatic versioning/publishing system to work. I've added it as an issue here for lack of a better place to record these thoughts. A couple of preliminary remarks:

• I've described this in terms of SemVer to match what we do currently, but most of the below is completely agnostic to the versioning system we use.
• I've used strings like Bump-Version: major just to have something concrete to talk about. I have no real opinions on the naming of things here.

I envision the workflow as consisting of three actions.

1. Check the PR is annotated correctly

To be run whenever a PR is created or its body (aka description) is edited.

The key points are:

• The PR body should be thing that gets annotated with magic strings to bump versions:

  • The PR is the unit of change and it makes sense to think of the level of change introduced by the PR as a whole, rather than assigning the changes (often arbitrarily) to individual commits.
  • The PR description is trivial to edit after creating the PR (even for non-technical contributors) without having to rebase and force push.
  • The PR description is immediately visible, without having to go hunting through the commit list looking for the prefix with the highest change level.

• We should use the git-trailer format for our magic strings:

  • Git already has a syntax for adding metadata to commit bodies, with some tooling support, so we might as well use it.
  • These go at the bottom of the PR description and would look something like:
    Bump-Version: major
  • This avoids wasting limited space in titles.

• The action should fail the PR if it doesn't have a trailer with a valid value.

  • Obvs.

• If the action doesn't find a matching trailer at all, it should insert a placeholder:

  • Suppose we're using the trailer Bump-Version:. If the action doesn't find such a trailer in the PR body it should insert into the PR body something like:
    Bump-Version: major/minor/patch/none (choose one)
  • This would cause the check to fail (because that's not a valid value for our Bump-Version trailer) but it should be immediately obvious to the PR author how to fix it without having to remember the syntax.

2. Tag new versions

This could be run on every merge to main, as we currently do. Or for some projects (opensafely-cli perhaps?) we might want to make it a manually dispatched workflow so we can control when we cut a new release.

This would need to:

• look at the commits between the last tagged version and the current HEAD;
• extract all trailers;
• compute the new version based on the change levels found (which might be no change);
• if there is a change, tag HEAD with the new version.

3. Ensure latest version is published

The key thing here (especially given the flakiness of Github's CI) is to make this an idempotent ratchet, rather than anything stateful. It's job is:

• find the latest version tag;
• check whether that specific version is already published in the appropriate place (Docker registry, PyPI, whatever);
• if it is, we're done;
• otherwise, checkout that specific tag;
• build and push (or better yet, retrieve some pre-built artefact and push);
• if we get an error saying "version already exists" then treat that as a success (in theory we've checked for this already, but in practice there's often a delay between a package being pushed and the version being available and we don't want spurious failures).

We'll obviously want to run this whenever we tag a new version. But there's no real harm in running it more often (e.g. every merge) as it's safe by design. It should also be manually dispatchable to allow us to retry following inevitable flakiness.

No files or tests are named. Start by reviewing the existing automatic versioning and publishing workflows, then compare them with the three proposed actions: PR trailer validation, version tagging, and idempotent publication; done means the desired workflow is designed and its behavior is agreed.