[CI] Use containers to cache dependencies to reduce the effect of intermitant service issues in CI runs
@machshev is already working on this.
Since Oct 1, 2025.
- Dominant language
- SystemVerilog
- Stars
- 3.6k
- Forks
- 1.1k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 141
Description
### Description
At the moment our CI jobs build the whole environment from a vanilla Ubuntu base image (Github runner VM). The benefit of doing this is that a PR can change the dependencies in the same PR i.e. add a python package, or Bazel dependency. This means that we are hitting 3rd party repos a lot during each PR CI run.
The proposal is to cache some of this within a container image, which would be rebuilt if the dependencies changed, on a successful merge to master. This means that for any PR that doesn't modify external dependencies there is no need to go fetch those dependencies. For PRs that do change dependencies the CI job would have to fetch just the additional packages/resources that are required.
- [ ] Review the existing CI jobs and work out which dependencies need caching
- [ ] Develop a proof of concept container image and evaluate
- [ ] Add a workflow to build and publish the container image(s) to a container registry on merge to master - gated on an actual change to the dependency definition.
- [ ] Update the existing CI workflows to use the new container(s)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.