[EPIC] CI improvement: stay within the ASF runner budget, cut wall clock, and restore test coverage
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## Context
ASF Infra disabled GitHub Actions on apache/hudi on 2026-08-04 over CI runner-time consumption (discussion #19515). #19523 and #19514 are the interim fix — reset `required_status_checks` and trim the matrix — to get merging again within the [ASF GitHub Actions policy](https://infra.apache.org/github-actions-policy.html) (concurrency MUST ≤20, SHOULD ≤15; 4,200 runner-hours/week cap). The trim buys headroom by deleting coverage (`[CI-TRIM]` lanes, commented in place, not restored). This epic tracks getting to CI that stays within budget, is fast, and covers what we ship.
## Streams
**Unblock (in flight)** — merge the interim fix. #19525, #19526.
**Runner budget & concurrency** — get under the SHOULD target and stop paying for what a change doesn't touch. Possible directions: cap concurrency (#19527), cache deps/build output across jobs (#19529), path-filter scoped changes (#19530).
**Wall clock** — shrink time to first signal and to full green. Possible directions: rebalance jobs by measured duration (#19528), fix/quarantine flaky tests so retries don't double-pay (#19534).
**Test coverage** — restore and broaden coverage lost in the trim. Possible directions: per-version Spark coverage (#19531), compile-only for parked Flink 1.18–2.0 (#19532), bundle validation for Flink/Kafka Connect/metaserver (#19533) and Presto (#19468).
**Cleanup** — keep the CI config itself maintainable. Possible directions: drop `validate-ci-baseline` once no pre-trim PR is open (#19535), keep per-job contexts out of `required_status_checks`, restore `[CI-TRIM]` lanes opportunistically (cheap compile/smoke over full suites for older versions).
## Tracking
Watch weekly runner-hour usage against the 4,200 cap so a regression is visible to us before Infra.
## Related
- Discussion #19515
- Migrated JIRAs, kept for history: #14887, #15002, #16341, #16170, #15749, #16476
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the interim fixes in #19523 and #19514, then review the stream issues #19527–#19535 and the ASF GitHub Actions policy. Done means the selected CI changes keep weekly usage below the 4,200 runner-hour cap, reduce wall-clock time, and restore the coverage described in this epic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, java
- Domain
- ci-cd, devops, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100