bcgov / bcgov/tier2-pattern-test

Pack chaos suite run 2026-09-10

Open
#10 3 comments 0 reactions 0 assignees View on GitHub
bug documentation enhancement
Dominant language
Gherkin
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

# Pack chaos suite — run tracking

**Smoke repo:** `bcgov/tier2-pattern-test`
**Suite:** `patterns/chaos/` (catalog C01–C22)
**Driver:** `node patterns/chaos/driver.mjs --run `

Fill **Result**, **Evidence**, and **Pack PR** after each scenario. Evidence = issue/PR/workflow URLs from driver JSON output or manual attestation.

## Results

| ID | Scenario | Result | Evidence | Pack PR |
| --- | --- | --- | --- | --- |
| C01 | thin issue gets needs-detail | | | |
| C02 | well-formed issue is not needs-detail | | | |
| C03 | docs-drift reacts to API change without docs | | | |
| C04 | Demo fail CI triggers CI diagnose | | | |
| C05 | plan without features fails local gate | | | |
| C06 | src change without plan fails gate | | | |
| C07 | ready-for-agent before CP2 does not orphan a mergeable PR | | | |
| C08 | ready-for-agent on thin issue | | | |
| C09 | evidence-only impl-shaped PR fails gate | | | |
| C10 | strict placeholders fail checkpoint gate | | | |
| C11 | CP1 request-changes blocks plan/ready until re-approve | | | |
| C12 | CP2 request-changes blocks ready-for-agent until re-approve | | | |
| C13 | CP3 request-changes requires fix and re-approve before merge | | | |
| C14 | missing COPILOT_ASSIGN_TOKEN fails loud | | | |
| C15 | conflicted impl branch shows conflict not fake-green | | | |
| C16 | over-scope impl rejected at CP3 not silent merge | | | |
| C17 | broken collect-metrics fails workflow | | | |
| C18 | maintain: hotfix without spec | | | |
| C19 | intent-ready on thin issue | | | |
| C20 | thin vertical slice still green after chaos | | | |
| C21 | acceptance test missing provenance header warns | | | |
| C22 | evidence-only with Closes keyword fails | | | |

## FAIL → fix

When any **blocker** scenario is FAIL:

1. Paste driver output (`failures` array + evidence URLs) in a comment on this issue.
2. Open a pack issue in `ai-sdlc` targeting the scenario `pack_owner` (see `patterns/chaos/catalog/.json`).
3. Fix pack, merge, re-enrol smoke repo if enrol files changed.
4. Re-run failed id: `node driver.mjs --run `.
5. Re-run **C20** before closing the suite.
6. Link the pack fix PR in the **Pack PR** column above.

**Waivers** — Document here with rationale only for `documentary` scenarios or `blocked-external` (org/network). Do not waive pack logic FAILs.

## Notes

- Local summary: `patterns/chaos/results/summary.md` (after each driver run).
- Durable wiki synthesis: filed after first full run in `wiki/synthesis/pack-chaos-suite.md`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with patterns/chaos/driver.mjs and the patterns/chaos/ catalog, then run the listed driver command against the smoke repo for each scenario. Record each Result, Evidence, and Pack PR in this issue, review patterns/chaos/results/summary.md, and complete the wiki synthesis after the full run. Done means all C01–C22 are recorded and C20 is rerun after any blocker fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
ci-cd, testing-qa
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
63/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.