aws-samples / aws-samples/sample-autonomous-cloud-coding-agents
perf(ci): shard the CDK jest suite (jest --shard) — DEFERRED, follow-up to #363
- Dominant language
- TypeScript
- Stars
- 143
- Forks
- 46
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 20
Description
Parent: #363 (build-performance umbrella). Spun from #363's DoD by the resolve-open-issues orchestrator; **needs maintainer `approved` before implementation (ADR-003)**.
> **⚠️ Re-framed 2026-07-29 after measured review of PR #364 (@theagenticguy).** This lever was originally filed as "the biggest remaining win." **Measurement shows it is currently net-neutral-to-slower and should be DEFERRED behind cheaper levers.** The design below is kept because it is the part people get wrong; the *priority* is what changed.
## Status: DEFERRED (documented design, do not implement yet)
## Why deferred — the numbers moved
The original ~298s `//cdk:test` figure was stale. After PR #371 (disable Lambda bundling in CDK unit-test synths) the suite is **~125s** (136 suites) on the 4-core `merge_group` runner (runs `30412723686` / `30409171662` / `30400147506`, 2026-07-28/29). Substituting today's inputs into the doc's own `slice + fixed_overhead` formula (per-job overhead ~120–220s, and caches currently **miss on every run**):
| Shards | Slice | + overhead | Shard wall |
|---|---|---|---|
| 1 (today) | ~125s | n/a | whole build ~148–192s |
| 2 | ~63s | 120–220s | ~183–283s |
| 4 | ~31s | 120–220s | ~151–251s |
A 4-way shard lands **at or above the current entire build** — before counting the aggregate job's own overhead, extra runner minutes, and coverage-merge complexity. Two further reasons it cannot help yet:
- **Synth is the next binding constraint** (`//cdk:synth:quiet` finishes ~142s), so sharding `//cdk:test` below ~142s buys nothing until synth is also addressed.
- **Caches miss on every `merge_group` run** (`node_modules=miss, venv=miss, jest=miss`) — fixing cache hit-rate is cheaper and *lowers this lever's own crossover point*.
## Do these FIRST (they dominate sharding on current numbers)
1. **#685** — investigate the `Free Disk Space` step (47-67% of pre-build overhead; the largest addressable cost). Cutting it drops fixed_overhead ~120-220s toward ~60-100s.
2. **#677** — bump default runner (vertical scaling: `jest maxWorkers` scales with cores, no per-job overhead duplication). Strictly better than sharding until the largest single runner saturates.
3. **#678** — path-filtered builds (skip `//cdk:test` on non-CDK PRs entirely).
4. **#684** — investigate `yarn install --check-files` (install cost is cache-independent; fixing cache hit-rate saves ~zero — this is NOT a lead lever).
## Revisit trigger
Reconsider sharding **only when `//cdk:test` serial time exceeds ~250s** on the then-current runner AND vertical scaling (#677) is exhausted AND caches are warm — i.e. one runner's cores can no longer hold the needed parallelism, or the suite exhausts a runner's memory/disk. At that point horizontal fan-out becomes the sole remaining lever.
## Design (kept for when the trigger fires — these are the fragile parts)
- Matrix job runs N shards; the **aggregate job must keep the `compute_type: [agentcore]` matrix so its check-run name stays `build (agentcore)`** — the required context in ruleset `14980587`. A job named plain `build` emits check `build`, which does NOT match the required `build (agentcore)` context and **deadlocks the merge queue** (the failure #327 + build.yml's header warn about). This is a REQUIRED-CONTEXT change, not workflow-only.
- Adding a `shard` dimension yields check names `build (agentcore, 1)`, `build (agentcore, 2)`, … — do NOT mark individual shards required (multiplies required contexts).
- **Aggregate gate must use `always()` + a step-level check, NOT a job-level `if: contains(...)`** — `contains()` is not a status function, so `if: ${{ contains(needs.*.result,'failure') }}` silently resolves to `success() && contains(...)` (a contradiction) and the job never runs, admitting failing shards. Correct form:
```yaml
build:
needs: [build-shard]
if: ${{ always() }}
steps:
- name: Gate on shard results
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: exit 1
```
- Merge per-shard coverage before threshold enforcement so `coverageThreshold` still holds on the combined result.
- Run self-mutation / drift checks **once** in the aggregate job, not per shard.
## Secondary (non-wall-clock) benefits, if near the crossover anyway
- Retry only the failing shard instead of the whole suite; isolate a flaky suite's blast radius.
Refs #363
Contributor guide
Research direction
Read build.yml and the parent issue #363, then review the cited merge_group runs and the documented overhead, synth, and cache measurements. Do not implement while the issue is deferred; it is ready for reconsideration only after the stated timing trigger, vertical scaling, and warm-cache conditions are met and a maintainer approves it under ADR-003.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, typescript
- Domain
- build-system, ci-cd, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100