Stop using -march=native in CI
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
-march=native is causing two problems:
1) Caching of artifacts generated this way cause SIGILL when running on older processor
## Evidence
Three consecutive runs of the same fwide PR (https://github.com/llvm/llvm-project/pull/196157), same code:
| Run | Azure Region | Cache Hits | Cache Misses | Result |
|-----|-------------|-----------|-------------|--------|
|
[25512875679](https://github.com/llvm/llvm-project/actions/runs/25512875679/job/74876008545)
| westus3 | 9 | 5354 | PASS |
|
[25524024922](https://github.com/llvm/llvm-project/actions/runs/25524024922/job/74916241365)
| northcentralus | 5345 | 0 | CRASH |
|
[25524839613](https://github.com/llvm/llvm-project/actions/runs/25524839613/job/74965830435)
| westus | 5345 | 0 | CRASH |
The first run had a nearly empty cache and compiled everything locally
(0.17% hit rate). An intermediate [syscall-unistd
run](https://github.com/llvm/llvm-project/actions/runs/25517783708/job/74893495220)
in eastus then populated the cache with object files compiled for that
region's CPU. Subsequent runs on different hardware got 100% cache hits
and crashed because the cached `.o` files use instructions their CPUs
lack.
This was mitigated by https://github.com/llvm/llvm-project/commit/a33ba5f0948c2e6c365468a113a7d1ae5afb37b5
2) Inconsistency of builds
CI aims to be 100% reproducible, and having the output from the compiler change based on what machine is silently picked isn't best practice.
That said, we almost certainly want to test on the range of CPUs since clearly different machine code is being generated for each target. We should have adequate test coverage across the range of CPUs available to us.
The mitigation used almost certainly should be rolled back after we stop using -march=native
Contributor guide
Research direction
Start by locating the CI configuration that adds -march=native and reviewing commit a33ba5f0948c2e6c365468a113a7d1ae5afb37b5, along with the referenced fwide and syscall-unistd runs. Check how cached object files are keyed across Azure regions and how CPU coverage is currently exercised. Done means CI builds are reproducible and cached artifacts do not require unsupported host instructions, while CPU variation remains tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, github-actions
- Domain
- build-system, ci-cd, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100