garrytan / garrytan/gstack

Periodic lane has never been green in its v1.77 shape — residual churn ledger post-v1.78 (bands, exited-class, known flakes)

Open
#2,756 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

The v1.78.0.0 wave fixed the v1.76 AUQ-collapse regression (deterministic reviewCount=0 across 9 plan-review E2Es, both attempts) — verified across three pinned-container periodic dispatches on the wave branch. But the weekly periodic lane in its CURRENT shape (v1.77's 73-shard sharded runner + pinned claude 2.1.251 + EVALS_ALL census) has never been green, and the residual failures churn between runs:

| run | code | problems | notes |
|---|---|---|---|
| 33363624506 (weekly baseline) | v1.76.0.0 | 14/73 | 9 = deterministic AUQ collapse (the fixed regression); + overlay-harness (all 8 arms), carve-section-loading timeout, section-loading, setup-gbrain-path4 |
| 33445027170 (wave round 1) | wave @ c9ea7295 | ~11 | collapse gone (counts 5-8); band-edge fails (8>7, paired 5>4), D19 completion-summary race |
| 33449045263 (wave round 2) | wave @ 3e4092f8 | 9 | churned set; intermittent 0s on eng-batching; routing model-choice flake; third-party-actions 1/50 census flake |
| 33454329611 (wave round 3) | wave @ 2d1645d5 (echo-only) | 10 | round-1/2 drivers ALL PASS (ceo-finding, eng-batching, paired, auto-decide, section-loading); different members fail instead (design-floor, split-overflow, mode-routing, conductor-prose 1st-ever fail) |

Classification with receipts:
- **Pre-existing, reproduces without wave code:** skill-e2e-autoplan-chain fails on a clean v1.77.0.0 worktree locally ("Required phase markers missing. Saw: []") and failed in the v1.76 baseline (timeout, hits=[]). overlay-harness (all arms, settingSources:[] — gstack prose never in context), carve-section-loading timeout, and plan-ceo-review-section-loading failed in the baseline too.
- **Band-edge variance:** the AUQ finding-count/paired bands (4-7, 2-4) sit close to the live model's natural distribution; individual tests coin-flip near the edges and the failing SET churns run-to-run — the opposite signature of the fixed regression (which was deterministic across attempts).
- **Known flake classes:** skill-routing-e2e journey-code-review (model picked /review over /code-review; catalog text untouched by the wave), skill-e2e-third-party-actions (1/50 in census, passes 5/5 locally and in the PR gate lane).

Suggested direction (next wave / own PR): recalibrate the AUQ bands against a fresh N-run distribution on the pinned container, harden the `outcome=exited`/`hits=[]` startup class (it predates the wave), and re-baseline the periodic exclude ledger per its own policy. Per test/periodic-exclude-data.ts, entries need a reason + re-entry condition — this issue is the tracking pointer.

Contributor guide

Open the contributing guide

Research direction

Start with test/periodic-exclude-data.ts and the listed periodic runs, then reproduce skill-e2e-autoplan-chain on a clean v1.77.0.0 worktree in the pinned container. Compare a fresh N-run distribution for the AUQ and paired bands and inspect the pre-existing outcome=exited/hits=[] failures. Done means the bands, startup class, and exclude entries have documented reasons and re-entry conditions, with the periodic lane re-baselined.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.