blackjax-devs / blackjax-devs/tuningfork
Verdict-bearing full-distribution GT gate: per-dim W1/σ two-prong equivalence test
- Dominant language
- Python
- Stars
- 3
- Forks
- 0
- Avg merge
- 7h 24m
- Merged PRs (30d)
- 4
Description
Successor to the z-band work (#223). The mean-z point-estimate comparison is now advisory at ensemble scale (#226) because a point-null test hits the NHST trap — it rejects *any* fixed discrepancy (including the GT's own MC error) as resolution grows. This tracks the positive replacement: a verdict-bearing comparison of the full marginal distribution that is trap-free by construction.
**Design (ratified):** per-dim **Wasserstein-1 in GT-σ units** run as a two-sample **equivalence test** — two prongs:
- **max prong** — `max_d W1/σ` vs a *floor-of-max* null (catches concentrated bias; exact FWER)
- **frac prong** — fraction of dims over `max(floor_d, τ_sci)` with a bootstrap-calibrated, correlation-aware threshold (catches diffuse bias that max-only misses)
Floor is asymmetric in both effective sizes (`c_d·√(1/E_s+1/E_g)`, conservative `E = min(bulk,tail)-ESS`); **second-stage** (runs only after R̂/ESS/div pass — ESS estimators fail exactly when samplers struggle); per-dim k̂ guard routes heavy-tailed dims to trimmed-W1/PIT; whitened sliced-W1 as an advisory joint realm; SBC (per-parameter ranks) as the no-GT escape hatch. `τ_sci = 0.05σ` starting value (distributional re-derivation owed, non-blocking).
**Why it escapes the trap:** as resolution grows the floor → 0 but the fixed materiality bar stays — a clean sampler always passes, a biased one is caught *harder*. Decisive receipt: the honest-null the current gate FAILs at z=4.5 is correctly cleared (0.022σ, 0/144 dims over floor).
**Prerequisite (now met):** the max prong is resolution-bound on single-chain GT (floor-of-max 0.075–0.099σ > 0.05σ); it needs the multichain GT, which landed in #228.
**Open items:** τ_sci distributional re-derivation; floor-construction method pinned with a golden; joint-realm power (whitened sliced-W1 vs targeted Frobenius-Δcorr); W1-vs-KS/AD measured-power head-to-head.
Lands as a new module inside the post-#227 `_gate/` package.
— 🤖 Blackjax-devs AI TL
Contributor guide
Research direction
Start by reading the post-#227 `_gate/` package and the multichain GT work in #228, then inspect the current gate behavior described in the issue. The new module should implement the ratified two-prong W1/σ equivalence comparison and reproduce the honest-null receipt that currently fails; resolve the listed open items and validate the golden floor construction before considering it done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100