element-hq / element-hq/synapse
Upload Python wheels directly to PyPI through the "trusted publishing" mechanism
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
It's now possible to configure PyPI to mint short-lived (15m) auth tokens to "trusted publishers" (i.e. a specific GitHub Actions workflow on a repo), so that they can upload wheels directly. See the documentation: https://docs.pypi.org/trusted-publishers/
### Benefits
* Eliminates the need to `upload` wheels manually using the release script.
* A side-benefit is one can more easily release Synapse with a poor internet connection.
* Allows us to more easily separate less-important wheels from the main release process.
* Currently the release process is bogged down by MacOS runners taking a long time to start. We could change the workflow so that release assets are published automatically once Linux wheels are built, and again once MacOS wheels are finally built. Today, we'd have to ask the release person to run `upload` a second time on their dev machine.
* Speeds up Synapse releases.
* The less manual steps the better.
### Security
An alternative would be to mint a long-lived access token to PyPI and store this in a GitHub Actions secret. This works, but then allows that secret to be used in any workflow, or local user. The trusted publishing scheme means that only a *specific* workflow executing in a *specific* GitHub Actions environment can mint a valid token - and it only lasts for 15m.
This workflow should only be runnable by a new tag being created, which is already limited to only repository maintainers.
PyPI has a great write-up of the security considerations for using Trusted Publishing with GitHub Actions: https://docs.pypi.org/trusted-publishers/security-model/
### Plan
1. Add the [`release-artifacts.yaml` workflow](https://github.com/element-hq/synapse/blob/6c16734cf34f9229fc1f552bf8aa8b9f2e7a53ad/.github/workflows/release-artifacts.yml#L183-L215) as a Trusted Publisher to the [`matrix-synapse` PyPI package](https://pypi.org/project/matrix-synapse/).
1. Modify the `release-artifacts.yaml` workflow to add an extra job that makes use of PyPI's [pypi-publish](https://github.com/marketplace/actions/pypi-publish) GitHub Action. This job should only run on a tag being published.
1. Optionally, we could [Create a GitHub Actions environment](https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments) to limit the job from running unless a maintainer approves it. Though I think this is already gated by the job requiring a tag to be published.
1. Remove the `upload` step [from the release script](https://github.com/element-hq/synapse/blob/6c16734cf34f9229fc1f552bf8aa8b9f2e7a53ad/scripts-dev/release.py#L468-L530).
credit to @sandhose for banging the drum about this for months.
Contributor guide
Assessment
This issue has not been assessed yet.