allenai / allenai/reward-bench

Multiple-comparison correction for per-subset comparisons (best-of-23 inflates "leads on subset X")

Open
#270 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
736
Forks
99
PR merge metrics
No merged PRs in 30d

Description

### Summary
RewardBench scores each model across 23 subsets, and it's common to report the subset where a model leads. Because that's the best of 23 comparisons, the reported subset win is the maximum of many noisy numbers and is biased upward — so an uncorrected per-subset p-value overstates significance. A short note (or a small helper) applying a family-wise correction would help readers avoid over-reading subset-level differences.

### Why it matters — two real cases from the released results
Per-subset two-proportion test, then the best A-direction win corrected with Šidák, `1 − (1 − p)^23`:

| Case | Overall | Best-subset raw p | Šidák (K=23) | Read |
|---|---|---|---|---|
| Two ~tied models (0.5350 vs 0.5340) | tied | 0.009 | **0.19** | selection artifact — not significant |
| A is 0.4 pp *worse* overall | A worse | 0.00049 | **0.011** | genuine subset result, survives correction |

The pair is the point: the correction isn't a blanket "everything is noise" — one win survives, one doesn't, and which survives is independent of the overall ranking. Between two *identical* models at K=23 you expect ~0.6 "significant" subset wins by chance alone, so a single marginal subset win is the base rate, not a finding.

### Suggestion
- a one-paragraph caveat in the README/leaderboard docs about best-of-K selection across the 23 subsets, and/or
- a small helper that reports Šidák/Bonferroni-corrected per-subset comparisons.

Method is deterministic (per-subset two-proportion z-test, Šidák over the 23 subsets), and the numbers above are from the released results files. Happy to open a PR with the helper + a reproduction script if that'd be welcome.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.