oxidecomputer / oxidecomputer/omicron
CI speed should be more observable
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
I'm going to inspect the output of 03c7f12cf7a70ed7b360a310e7a6b46d0075ac72 , with a breakdown of "where are we spending our time".
I would like to automate the duration and tracking of each of these steps, so it's more obvious where optimization efforts should go.
Tasks:
- Identify the steps involved with the following jobs
- build-and-test
- build OS images
- build TUF repo
- Alter the jobs to emit more information summarizing "where time was spent"
- Start tracking this information over time
Breakdown
- build-and-test(helios): 1h 28m 19s
- build-and-test(ubuntu-22.04): 52m 33s
- buildomat / helios / build OS images: 1h 22m 53s
- buildomat / helios / build TUF repo: 1h 56m 14s
There are other tasks here, but this is meaty enough to dig into and start optimizing.
build-and-test (helios)
| Time | Seq | Description | How long? | Percentage of Total Time |
|---|---|---|---|---|
| 2023-11-08T14:46:46.396Z | 1 | Job Start, doing minimal setup | 4.197 seconds | 0.08% |
| 2023-11-08T14:46:50.593Z | 15 | Download cargo, rust (1.72.1) | 25.198 seconds | 0.48% |
| 2023-11-08T14:47:15.791Z | 59 | Authentication | 0.043 seconds | 0.001% |
| 2023-11-08T14:47:15.834Z | 61 | Clone and checkout | 7.026 seconds | 0.13% |
| 2023-11-08T14:47:22.860Z | 77 | Simply running cargo --version downloads another rust toolchain (1.73.0), and then we download nextest too |
25.275 seconds | 0.48% |
| 2023-11-08T14:47:48.135Z | 121 | Using pkg to download prerequisites (build-essential, llvm, xml, etc) | 96.942 seconds | 1.85% |
| 2023-11-08T14:49:25.077Z | 263 | Download cockroach, clickhouse, console, maghemite, dendrite, tranceiver control | 35.246 seconds | 0.673% |
| 2023-11-08T14:50:00.323Z | 713 | Downloading third-party dependency crates | 35.931 seconds | 0.686% |
| 2023-11-08T14:50:36.254Z | 1435 | Compiling third party crates | 600.307 seconds | 11.47% |
| 2023-11-08T15:00:36.561Z | 3663 | Compiling our crates | 2148.442 seconds | 40.69% |
| 2023-11-08T15:36:05.003Z | 4024 | Start running nextest, which appears to be downloading and compiling more 3P crates | 13.441 seconds | 0.257% |
| 2023-11-08T15:36:18.444Z | 4233 | Run crdb-seed (which appears to also compile from scratch ) | 232.27 seconds | 4.44% |
| 2023-11-08T15:40:10.714Z | 4562 | Run crdb-seed | 6.455 seconds | 0.123% |
| 2023-11-08T15:40:17.169Z | 4571 | Run tests | 1897.63 seconds | 36.277% |
| 2023-11-08T16:11:54.799Z | 5491 | Build doctest | 65.025 seconds | 0.096% |
| 2023-11-08T16:11:59.824Z | 6315 | Run doctest | 5.025 seconds | 2.244% |
| 2023-11-08T16:13:57.214Z | 6861 | Finished running doctest | N/A | N/A |
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting commit 03c7f12cf7a70ed7b360a310e7a6b46d0075ac72 and the referenced build-and-test run. Identify the definitions for the build-and-test, build OS images, and build TUF repo jobs, then determine how each step can report duration. Done means the jobs summarize where time was spent and that information is tracked over time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ci-cd, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100