CI: reduce test wall time in GitHub Actions and Buildkite

CI wall time has grown to the point where it shapes what changes are worth making. On #174 the
`gfortran-12 coverage` job was **cancelled twice at its 90-minute timeout**, and the second
cancellation was triggered by a commit that changed nothing but documentation. That is not a
one-off: it cost a round of work trimming tests that turned out not to be the cause, and a second
round putting them back after the trim lost real coverage.

This issue is about the runtime itself, in both GitHub Actions and Buildkite. All numbers below are
measured, and I have flagged which are CI-observed and which are local.

## The shape of the problem

**Observed `gfortran-12 coverage` durations on one branch, against `timeout-minutes: 90`:**
43, 82, 85, 85, 90 (cancelled), 90 (cancelled), 85 min. The 43-minute run carried *more* tests than
the 82-minute one. So **runner speed dominates, not test count** — the job sits close enough to the
cap that a slow draw cancels it regardless of content. Debug jobs land at 54–84 min; release jobs at
16–21 min; Buildkite at ~61 min.

**Where the time actually is** (289 tests, `CTestCostData.txt`, gfortran debug build on a
workstation — debug is the relevant configuration since it is the slow one in CI):

| | tests | cost | share |
| --- | --- | --- | --- |
| total | 289 | 7723 s (129 min) | |
| slowest single test | 1 | 813 s | **10.5%** |
| top 10 | 10 | 4065 s | **53%** |
| ≥ 20 s | 55 | 7342 s | **95%** |
| < 5 s | 217 | 95 s | 1.2% |

By label: `example` 2787 s across 17 tests, `serial` 4594 s across 226, `parallel` 1161 s across 46.

The ten worst:

```
813 s linear_euler3d_spherical_soundwave_radiation
478 s linear_shallow_water_2d_constant
421 s linear_shallow_water_2d_nonormalflow
412 s linear_shallow_water_2d_radiation
368 s advection_diffusion_3d_rk4
355 s ec_advection_3d_rk3
343 s linear_euler2d_planewave_reflection
332 s linear_euler2d_planewave_propagation
276 s advection_diffusion_3d_nostress
268 s advection_diffusion_3d_rk3
```

**The suite is executed 11 times per pull request.** Eight GitHub Actions jobs — `linux-gnu-cmake`
(gfortran 9/10/11 debug + gfortran-12 coverage) and `linux-gnu-multithreaded-cmake` (gfortran
9/10/11/12 release) — plus three Buildkite pipelines (MI210, V100, x86 CPU). Every one of them runs
the whole thing.

## Optimizations, roughly in order of leverage per unit of risk

**1. Run ctest in parallel. Nothing currently does.** Every invocation in
`.github/workflows/*` and `.buildkite/*` is bare `ctest` with no `-j`, so 289 tests run
one at a time on a multi-core runner while `make -j $(nproc)` is used two steps earlier. This is the
single largest free win.

*Prerequisite, and the reason it cannot just be switched on:* `add_mpi_fortran_tests` in both
`test/CMakeLists.txt` and `examples/CMakeLists.txt` sets `LABELS` only — no `PROCESSORS` property.
`ctest -j` would therefore schedule many 2- and 4-rank MPI tests concurrently and oversubscribe the
runner, which at best inflates the times it is meant to reduce and at worst makes MPI tests flaky.
`PROCESSORS` is currently set on 7 tests out of ~48 MPI ones. Setting it in the two helpers is a
small, contained change and should land first.

**2. Stop running the full example set in every configuration.** `example`-labelled tests are 2787 s
of the 7723 s — 36% — across 17 tests, and they are integration runs whose value is largely
compiler-independent. Running them in one configuration (say the coverage job, which feeds codecov)
and excluding them elsewhere with `-LE example` would take a large bite out of seven of the eight
Actions jobs.

**3. Trim the heaviest individual tests.** One test is 10.5% of the suite and ten are 53%. Most are
long time-integration runs, and it is worth asking per test whether the step count still buys
anything the assertion needs — the tail is already cheap, so this is where the remaining time is.
There is precedent: `d88910d4` trimmed the 3-D AMR soundwave family for exactly this reason.

**4. Reconsider whether three debug matrix entries need the whole suite.** gfortran 9/10/11 debug
exist to catch compiler-compatibility problems, and those surface at *compile* time or in a smoke
subset. One full debug run plus a build-and-subset for the others would cut a lot. (Related, and
worth deciding separately: `linux-amdflang-cmake` and `linux-nvidia-hpc-cmake` are
`disabled_manually`, so of the four compilers CLAUDE.md requires, CI currently verifies one. Whatever
is done here should leave room to re-enable those rather than assuming gfortran is the whole matrix.)

**5. Drop `--verbose` from the Actions ctest calls.** `--output-on-failure` is already passed and is
what you want; `--verbose` additionally dumps every passing test's output, which for 289 tests is a
large log for no diagnostic gain.

**6. Raise `timeout-minutes` as a stopgap only.** All four build/test workflows use 90. Raising it
would stop the spurious cancellations immediately, but it treats the symptom — worth doing alongside
the real fixes rather than instead of them.

## Buildkite specifically

The three GPU/CPU pipelines each build from scratch and run the full suite (~61 min end to end).
Items 1–3 apply unchanged. Two additional questions: whether the MI210 and V100 pipelines need the
entire suite rather than the subset that actually touches device code paths, and whether the `x86
CPU` pipeline is duplicating what the eight Actions jobs already cover.

## Suggested sequencing

`PROCESSORS` on the MPI helpers → `ctest -j` everywhere → `-LE example` on the jobs that do not feed
codecov → then measure again before touching individual test costs. Steps 1 and 2 are mechanical and
should be measured together, because the concentration figures above mean parallelism and example
exclusion interact: with `-j`, excluding a 46-minute serial block matters less than it looks.

## One measurement caveat

The cost table is from a debug build on a 12-core workstation, not a CI runner. Debug is the right
configuration to reason about (release runs the same suite in 16–21 min, so the heavy
time-integration tests are roughly 4–5× slower under `-O0` plus bounds checking), but absolute
numbers will differ on a 2- or 4-core runner and the *relative* concentration is the durable part.
Before and after any change here, `Testing/Temporary/CTestCostData.txt` from the CI run itself is
the number to compare.

Start with add_mpi_fortran_tests in test/CMakeLists.txt and examples/CMakeLists.txt, then inspect the bare ctest calls in .github/workflows/* and .buildkite/*. Use Testing/Temporary/CTestCostData.txt and CI timings as the baseline. Done means the agreed parallelism and test-selection changes reduce wall time without MPI oversubscription or lost coverage.

Written by the indexing model from the issue text.