certbot / certbot/certbot

[Task]: test tools/finish_release.py as part of CI

Open
#10,523 0 comments 0 reactions 1 assignee Claimed by @bmw View on GitHub
Dominant language
Python
Stars
33.2k
Forks
3.5k
Avg merge
6d 10h
Merged PRs (30d)
3

Description

### Task description

in the last planned release, tools/finish_release.py broke. see https://github.com/certbot/certbot/pull/10503. to help prevent this in the future, we planned to start running this script as part of our nightly CI so we can catch potential problems before release time. this is the issue i said i'd write about this tracking that work

i'm open to other options, but i was imagining we'd do this by:

1. give the credential described [here](https://github.com/certbot/certbot/blob/17a1f0e1144db28f119b28ad56dca25e79c50a95/.azure-pipelines/templates/jobs/common-deploy-jobs.yml#L17-L38) access to the candidate, beta, and edge channels
2. update finish_release.py to do more than just move release snaps between the beta and stable channels. my first thought on how to do this is to use optional CLI flags which we'd set in CI but not require during the release for convenience. there's at least a few pieces to this which i see right now:
1. we'd need to be able to find the snap we want to move which i'd recommend we accomplish by setting a `--target-version` flag to a value that'd be something like 5.3.0.dev0. this would cause the script to look for that version in the snap store rather than querying azure pipelines. if this flag isn't set, it'd query azure pipelines which i think is the behavior we want when running the script locally for a release. alternatively, since testing the azure pipelines querying code in CI is difficult, we could remove this functionality and make `--target-version` a required argument to finish_release which would also remove finish_release.py's only non-stdlib dependency. if this alternate approach is taken, we also need to update the release instructions
2. we'd also need to specify which channel we want to move snaps from through a flag named something like `--source-channel`. i'd recommend this defaults to "candidate", but in CI we'd set it to "edge".
3. snaps are moved up one channel. that is, edge goes to beta, beta goes to candidate, and candidate goes to stable. i don't foresee us using the beta to candidate functionality, so that could be skipped if it's more convenient
3. as part of the [nightly deploy stage](https://github.com/certbot/certbot/blob/17a1f0e1144db28f119b28ad56dca25e79c50a95/.azure-pipelines/templates/stages/nightly-deploy-stage.yml) (but not the [release deploy stage](https://github.com/certbot/certbot/blob/17a1f0e1144db28f119b28ad56dca25e79c50a95/.azure-pipelines/templates/stages/release-deploy-stage.yml)), we set up and run finish_release.py to move recently release edge snaps to the beta channel. this will require installing finish_release.py's dependencies and blocking this job on the current snap deploy jobs
4. we update the [release deploy stage](https://github.com/certbot/certbot/blob/17a1f0e1144db28f119b28ad56dca25e79c50a95/.azure-pipelines/templates/stages/release-deploy-stage.yml) to move snaps to the candidate channel
5. profit?

this may not be a complete plan and i may be missing things here, but i believe an approach like this would allow us to run and test the script in CI nightly

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.