Second rater for Budapest District V: the rubric is now confounded with the benchmark's only ranking inversion

Why this is now the top open item in the model comparison

Budapest District V's ground truth was reviewed on 2026-07-27 and the reviewer rated their own pass low confidence — the US-derived curb-ramp rubric does not transfer cleanly to Hungarian street furniture. That was logged as a hygiene follow-up at the time (PR #64).

The full 8-model challenger run on 2026-07-28 (#73) promoted it from hygiene to blocking, because Budapest produced the benchmark's only ranking inversion and the low-confidence GT is one of exactly two explanations for it.

The result that needs explaining

Qwen-32B falls below Qwen-8B. On all five other cities 32B leads 8B by 0.05–0.07 without exception. Nothing else in the ordering moves.

It did not fail — it stopped firing: 30 detections across 125 panos (0.24/pano vs 1.0–1.2 on the US cities), zero abstentions, both GPUs loaded, all 125 panos scored, clean job log. Its null recall is 0.002, so there is no chance recall available to explain the number away.

Morgantown is the control and rules out a bad checkpoint or a broken leg: same camera family, same launcher, same day, Qwen-32B scores a completely ordinary F1 0.421.

The two explanations, and why this split can't separate them

1. Domain. Qwen-32B genuinely cannot read Hungarian streetscapes.
2. Rubric. Qwen-32B and this reviewer disagree about what counts as a curb ramp.

Qwen-32B is the most conservative model in the roster everywhere — it always trades recall for precision. A ground truth that disagrees about where the class boundary sits will punish a model that only fires when confident far harder than one that sprays boxes and collects the overlap. Both hypotheses predict this exact table.

The specific known rubric ambiguity is the likely lever: Budapest corners often carry one broad diagonal ramped apron serving two crossing directions. The reviewer generally counted these as two ramps; that single call alone is worth ~4 points of precision (0.873 default vs 0.910 with --lenient-duplicates).

What to do

A second independent rater over the same 125 panos, using scripts/gt_gallery.py benchmark/budapest_district5 at model resolution (same protocol as every other split).

• Record the second rater's own confidence in review_notes, as #64 added.
• Explicitly capture a call on the diagonal-apron case — one ramp or two — since it is the largest single known source of disagreement.
• Compare per-detection verdicts, not just the aggregate P/R. The question is whether the disagreements cluster on the panos where Qwen-32B declined to fire.

This is a labeling task, not a compute task. Running more models on Budapest will not resolve it — that has already been done. It is the only route to separating the two hypotheses.

What is not at stake

The headline claim is unaffected either way: RampNet's margin over the best challenger here is 0.263, mid-band against 0.19–0.34 on every other split, so it survives ground truth the reviewer distrusts. What is at stake is whether any between-challenger ordering on this split means anything, and whether the benchmark can make a non-US claim at all.

Until this is resolved, docs/model_comparison.md frames Budapest as a rubric-robustness test, not a difficulty test, and bars its numbers from being pooled with the US splits.

Also still open on this split

scripts/build_benchmark_dataset.py is hardcoded to bend+richmond, so clovis/morgantown/budapest/annapolis and the review_notes fields never reach the HF benchmark dataset (part of #21).

Refs: #64 (split + review), #73 (challenger results), #20 (harness).

Start with scripts/gt_gallery.py and the benchmark/budapest_district5 split, using the model-resolution protocol described in the issue. Rate all 125 panos independently, record confidence in review_notes, and explicitly resolve diagonal aprons as one ramp or two. Compare per-detection verdicts, especially where Qwen3-VL-32B did not fire, and report whether disagreements cluster there.