linebender / linebender/vello

Improve CI performance

Open
#1,357 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

testing
Dominant language
Rust
Stars
4.3k
Forks
300
Avg merge
2d 13h
Merged PRs (30d)
61

Description

Our CI is extremely slow; for example, for 184d6416521d348380563c933e245766a2123647, the CI run took 13 minutes (!). See https://github.com/linebender/vello/actions/runs/20951342050 (this link will expire in 30-90 days). There are, unfortunately, two long poles in the tent there.

# Clippy Jobs

The clippy jobs took: 7 minutes for Ubuntu, 9 minutes for macOS and 12 minutes for windows.

This is due to the cargo hack checks, which have 75 configurations to go through (+another 75 for tests/examples). Ideally, we'd find a way to lower that number. Incidentally, we're not even checking the powerset.

# Test jobs

The test jobs took:

- 7 minutes on Ubuntu. That's 1 minute of compiling, and 6 minutes of runtime.
- 8 minutes on macOS. That's 3 minutes of compiling (seems to have had a cache miss; worth factoring in?), and slightly less than 5 minutes of runtime.
- Some amount on windows, my point remains.

Theoretically, the length of test runtime could be improved by running them in release mode.
For reference, on my machine, the tests take 53 seconds to run in debug mode, vs 6 seconds in release mode.

However, note that obviously we'd need to actually perform the benchmarking to see if the extra compile time cost is worthwhile..


For a sense of how impactful this can be on processing times, consider that I've written up this full issue in the time waiting for #1348 to land!
See also the Zulip thread [#vello > Speeding up Vello CI](https://xi.zulipchat.com/#narrow/channel/197075-vello/topic/Speeding.20up.20Vello.20CI/with/562427534)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked GitHub Actions run and compare the Clippy cargo-hack matrix with the test jobs. Benchmark release-mode tests against any added compile time, then evaluate reducing redundant configurations or changing test execution. Done means a measured CI-time improvement without losing coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust
Domain
ci-cd, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.