microsoft / microsoft/vcpkg

Improvement to CI speed, for expected failed builds

Open
#52,192 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
CMake
Stars
27.5k
Forks
7.7k
Avg merge
2d 19h
Merged PRs (30d)
321

Description

**Describe the bug**
in the ci, when a port has Succeeded and nothing has changed in its port hash, the system looks up a previous runs compiled results from a cache and uses them. If the build fails, the system will always try a rebuild every ci run, even if the build is marked as expected to fail.

It would be nice if a failure cache item was saved on failure with a specific port hash, so that reruns of expected fails of ports did not take time to run, and we get a fast path.

This would save time and money on slow ci runs on ci ( libtorch, full ci is 12 hours, repeated ci is > 1hour)

**Environment**
- OS: CI
- Compiler: ALL

**To Reproduce**
Steps to reproduce the behavior:
have a pr on libtorch, with all checks passed, change the status from draft to ready to review, all checks are run again, and will take 3 or more hours to rerun the expected failed cuda debug builds.

**Expected behavior**
Each run would know that that specific hash of the port has run before with no changes, and the cached result used,

**Failure logs**
Feature Test [15/17] libtorch[core,vulkan]:x64-windows
Skipping testing because the ABI hash e0edb3fd4e5acc588806d37443776e8f4f2c8d89f11b755dfa9600d646301412 is already in the binary cache.
Feature Test [16/17] libtorch[core,cuda]:x64-windows
-- this is not skipped because it fails, and it failed last time with exaclty the same inputs, so its failure should be cached just as success is cached. -- especially as it is expected to fail in the baseline.

**Additional context**
I think this could save a huge amount of time in some build scenarios and would likely save a lot of ci build hours.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the CI port build path that reads successful results from the binary cache, focusing on the libtorch expected-failure CUDA debug builds and their port hash. Add coverage for repeated expected failures with unchanged inputs, and verify that the cached failure is reused instead of rebuilding on later CI runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, ci-cd
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.