apache / apache/hudi

[EPIC] CI improvement: stay within the ASF runner budget, cut wall clock, and restore test coverage

Open
#19,524 0 comments 0 reactions 0 assignees View on GitHub
area:ci priority:high type:epic
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

## Context

ASF Infra disabled GitHub Actions on apache/hudi on 2026-08-04 over CI runner-time consumption (discussion #19515). #19523 and #19514 are the interim fix — reset `required_status_checks` and trim the matrix — to get merging again within the [ASF GitHub Actions policy](https://infra.apache.org/github-actions-policy.html) (concurrency MUST ≤20, SHOULD ≤15; 4,200 runner-hours/week cap). The trim buys headroom by deleting coverage (`[CI-TRIM]` lanes, commented in place, not restored). This epic tracks getting to CI that stays within budget, is fast, and covers what we ship.

## Streams

**Unblock (in flight)** — merge the interim fix. #19525, #19526.

**Runner budget & concurrency** — get under the SHOULD target and stop paying for what a change doesn't touch. Possible directions: cap concurrency (#19527), cache deps/build output across jobs (#19529), path-filter scoped changes (#19530).

**Wall clock** — shrink time to first signal and to full green. Possible directions: rebalance jobs by measured duration (#19528), fix/quarantine flaky tests so retries don't double-pay (#19534).

**Test coverage** — restore and broaden coverage lost in the trim. Possible directions: per-version Spark coverage (#19531), compile-only for parked Flink 1.18–2.0 (#19532), bundle validation for Flink/Kafka Connect/metaserver (#19533) and Presto (#19468).

**Cleanup** — keep the CI config itself maintainable. Possible directions: drop `validate-ci-baseline` once no pre-trim PR is open (#19535), keep per-job contexts out of `required_status_checks`, restore `[CI-TRIM]` lanes opportunistically (cheap compile/smoke over full suites for older versions).

## Tracking

Watch weekly runner-hour usage against the 4,200 cap so a regression is visible to us before Infra.

## Related

- Discussion #19515
- Migrated JIRAs, kept for history: #14887, #15002, #16341, #16170, #15749, #16476

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the interim fixes in #19523 and #19514, then review the stream issues #19527–#19535 and the ASF GitHub Actions policy. Done means the selected CI changes keep weekly usage below the 4,200 runner-hour cap, reduce wall-clock time, and restore the coverage described in this epic.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, java
Domain
ci-cd, devops, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.