containerd / containerd/nerdctl

[CI]: CI should finish under 15 minutes

Open
#4,193 0 comments 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Go
Stars
10.4k
Forks
826
Avg merge
1d 23h
Merged PRs (30d)
44

Description

### What is the problem you're trying to solve

CI runs take a long time.
This is of course getting increasingly slower as we introduce more tests (and/or more exotic targets).

While the general trend is inevitable, we should take a proactive approach to counter it.

With the recently introduced toy-analytics in jobs summary, we will gain insight into what are our key slowdowns, from which we could derive things to do maximizing for impact.

### Describe the solution you'd like

From early feedback on the analytics, the long poles (unsurprisingly) at this point are `container` and `compose` "flaky/legacy" tests - each of these two are taking about 9 minutes:

Image

Possible action items:
- [ ] long-term objective is clearly to rewrite `container` and `compose` "flaky bucket" to use the new test framework - it is hard to provide an accurate estimate on the actual gain, but it is likely over 25%
- [ ] non-flaky, flaky are two steps running in sequence - if we are ready to accept an increase in the number of pipelines on GHA, we could instead make these parallel - the net gain would be the current "non-flaky" duration (about 5 minutes)
- [ ] in the same train of thought: `flaky` tests are running in one "bucket" - if we would split them out in `compose` vs. `container + others`, we could gain about 9 minutes
- [ ] testing out-of-container _could_ speed things up - this requires quite a lot of work to have a reliable provisioning mechanism "on-host"
- [ ] work in progress "proofing" project to mock third-party services and images could deliver a nice speed-up as well (on top of increased reliability)

Note that steps we take to speed-up the CI should be carefully thought out against increased complexity though.
It is paramount that testing remains simple on developers laptops, and that github workflows do not become too convoluted.

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.