model-drift.md cites a fabricated single-run gap/p-value for unfair.py (6.39%/p=0.348 instead of the real 7.16%/p=0.2564)
@Peganofred is already working on this.
Since Sep 9, 2026.
- Dominant language
- HTML
- Stars
- 47
- Forks
- 45
- Avg merge
- 1h 57m
- Merged PRs (30d)
- 95
Description
Where: explainers/model-drift.md - "The original unfair.py audit reports a single 6.39% gap from one random 80/20 split, not statistically significant at n=1,000 (p=0.348)."
The gap: the real, deterministic (random_state=42) output of German Credit Lending/unfair.py is a 7.16% gap with p=0.2564 - not 6.39%/p=0.348.
Repro:
$ python3 "German Credit Lending/unfair.py"
Fairness Gap: 7.16%
Permutation test p-value: 0.2564 (not statistically significant at α=0.05)
Ran 3 times - byte-identical output each time (the script is fully seeded). The correct 7.16% figure is also what's cited elsewhere in this same repo for this exact script (explainers/reject-inference.md, explainers/selection-bias.md, llms-full.txt) - model-drift.md is the only file citing 6.39%/p=0.348 for it.
Why it matters: the rest of the explainer's own table (5 sequential rolling-window rows: 4.3/11.7/13.6/15.1/10.2%) independently reproduces correctly and is internally consistent - only this one anchor sentence is wrong, giving a reader a false single-run number to compare the rolling-window swings against.
Suggested fix: replace "6.39%... p=0.348" with "7.16%... p=0.2564", matching the script's actual (and elsewhere-correctly-cited) output.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.