Hand-label approximate screen_tr (and screen_tl) on ~40 frames for TR-projection validation
@hans is already working on this.
Since May 24, 2026.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Follow-up to #8 / PR #9.
## Motivation
PR #9 added `big_star` as a 5th anchor in the per-frame H fit. Empirical run on EC347 showed the TR_x catastrophic outlier magnitude dropped ~4.5× (max 2.76M px → 606K px, outlier rate 4.6% → 1.6%), but did NOT eliminate them. Root cause: the 5 anchors are still all clustered in the lower portion of the iPad screen (`box_bl_screen ≈ (6, 561)`, `box_br_screen ≈ (278, 564)`, big_star wherever the active dot is — typically not at the top), so projecting `screen_tr = (2388, 0)` is still extrapolation.
**The diagnostic we don't have:** how badly is the current 5-anchor H actually off at TR in *frame-pixel reality* (vs. wild numerical projections that may or may not be visually obvious)? A handful of approximate TR labels would answer this directly and would tell us whether further work on TR (multi-star anchors, weighted DLT, dedicated TR detector) is worth the investment.
## Ask
Hand-label `screen_tr` (and `screen_tl` if visible enough to estimate) at roughly **40 frames** spread across the EC347 recording, using the existing `notebooks/label_homography_correspondences.py` labeling UI. Mark them as `quality=\"approximate\"` (TR/TL are intrinsically approximate due to photodiode occlusion — same convention as the existing few approximate-TR labels).
### Sampling pattern
Aim for diversity in head pose / camera framing, not concentration in any one regime:
- **~25 frames evenly spaced** across the full video (every ~1200 frames covers 30K-frame video).
- **~10 frames at the known-bad TR-extrapolation frames** identified by PR #9's diagnostic (the 26 frames where v2 still has `|projected TR_x| > 100K`). A sample list:
- 20097-20113 (cluster of 6+ frames, same head pose presumably)
- 11507-11519 (cluster of ~6)
- 31299-31301 (cluster of 3)
- 5612, 9614, 10298, 10320, 15869, 22223-22224, 24488
- Full list available in `results/EC347/phase1c_per_frame.parquet` — filter `n_anchors_used==5 & abs(TR_x_projected) > 100000`.
- **~5 frames in the existing 26-frame labeled set** (frames 664, 685, 731, 1550, 2189, 5811, 6104, 9124, 9432, 18973, 19671, 30950, etc.) — useful to cross-check labeling consistency over time and double-check that the existing TR labels (where present) are still in the ballpark.
## Output
Updated `results/EC347/homography_labels.parquet` with new rows of `label_type=\"screen_tr\"` (and optionally `\"screen_tl\"`). The labeling notebook handles the schema.
## Then what
PR #9 follow-up notebook (out of scope for this issue) will:
1. **Diagnostic**: project `screen_tr=(2388, 0)` through the saved 5-anchor H at each labeled frame; report distribution of frame-px residuals. If median > 50 px, that's empirical evidence the 5-anchor fit's TR is meaningfully off (not just numerically wild).
2. **Optional 6-anchor fit**: at the ~40 labeled frames, refit with screen_tr included; measure how much it changes the H and whether downstream (Phase 2) metrics improve at those frames.
3. **Future**: if (1) shows large residuals AND visual inspection of overlays at those frames shows enough TR pixel structure to detect automatically, this seeds a future TR-detector spec (analogous to the box-corner detector in `specs/phase1c-box-corner-detector.md`).
## Out of scope
- A dedicated screen_tr detector (Phase 1d?). Needs the labels first to decide if it's even feasible.
- Re-running the existing 4-anchor calibration / per-frame fit. The new labels feed validation, not the existing pipeline directly.
- Labeling NP168 or other subjects — EC347 only for now.
## Approximate time investment
The labeling notebook lets you scrub a video and click corners. ~40 frames at ~30-60 s each (including frame navigation) ≈ 30-45 minutes total.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.