hans / hans/dot-prediction

Hand-label approximate screen_tr (and screen_tl) on ~40 frames for TR-projection validation

Open
#10 0 comments 0 reactions 1 assignee View on GitHub

@hans is already working on this.

Since May 24, 2026.

Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Follow-up to #8 / PR #9.

## Motivation

PR #9 added `big_star` as a 5th anchor in the per-frame H fit. Empirical run on EC347 showed the TR_x catastrophic outlier magnitude dropped ~4.5× (max 2.76M px → 606K px, outlier rate 4.6% → 1.6%), but did NOT eliminate them. Root cause: the 5 anchors are still all clustered in the lower portion of the iPad screen (`box_bl_screen ≈ (6, 561)`, `box_br_screen ≈ (278, 564)`, big_star wherever the active dot is — typically not at the top), so projecting `screen_tr = (2388, 0)` is still extrapolation.

**The diagnostic we don't have:** how badly is the current 5-anchor H actually off at TR in *frame-pixel reality* (vs. wild numerical projections that may or may not be visually obvious)? A handful of approximate TR labels would answer this directly and would tell us whether further work on TR (multi-star anchors, weighted DLT, dedicated TR detector) is worth the investment.

## Ask

Hand-label `screen_tr` (and `screen_tl` if visible enough to estimate) at roughly **40 frames** spread across the EC347 recording, using the existing `notebooks/label_homography_correspondences.py` labeling UI. Mark them as `quality=\"approximate\"` (TR/TL are intrinsically approximate due to photodiode occlusion — same convention as the existing few approximate-TR labels).

### Sampling pattern

Aim for diversity in head pose / camera framing, not concentration in any one regime:

- **~25 frames evenly spaced** across the full video (every ~1200 frames covers 30K-frame video).
- **~10 frames at the known-bad TR-extrapolation frames** identified by PR #9's diagnostic (the 26 frames where v2 still has `|projected TR_x| > 100K`). A sample list:
- 20097-20113 (cluster of 6+ frames, same head pose presumably)
- 11507-11519 (cluster of ~6)
- 31299-31301 (cluster of 3)
- 5612, 9614, 10298, 10320, 15869, 22223-22224, 24488
- Full list available in `results/EC347/phase1c_per_frame.parquet` — filter `n_anchors_used==5 & abs(TR_x_projected) > 100000`.
- **~5 frames in the existing 26-frame labeled set** (frames 664, 685, 731, 1550, 2189, 5811, 6104, 9124, 9432, 18973, 19671, 30950, etc.) — useful to cross-check labeling consistency over time and double-check that the existing TR labels (where present) are still in the ballpark.

## Output

Updated `results/EC347/homography_labels.parquet` with new rows of `label_type=\"screen_tr\"` (and optionally `\"screen_tl\"`). The labeling notebook handles the schema.

## Then what

PR #9 follow-up notebook (out of scope for this issue) will:

1. **Diagnostic**: project `screen_tr=(2388, 0)` through the saved 5-anchor H at each labeled frame; report distribution of frame-px residuals. If median > 50 px, that's empirical evidence the 5-anchor fit's TR is meaningfully off (not just numerically wild).
2. **Optional 6-anchor fit**: at the ~40 labeled frames, refit with screen_tr included; measure how much it changes the H and whether downstream (Phase 2) metrics improve at those frames.
3. **Future**: if (1) shows large residuals AND visual inspection of overlays at those frames shows enough TR pixel structure to detect automatically, this seeds a future TR-detector spec (analogous to the box-corner detector in `specs/phase1c-box-corner-detector.md`).

## Out of scope

- A dedicated screen_tr detector (Phase 1d?). Needs the labels first to decide if it's even feasible.
- Re-running the existing 4-anchor calibration / per-frame fit. The new labels feed validation, not the existing pipeline directly.
- Labeling NP168 or other subjects — EC347 only for now.

## Approximate time investment

The labeling notebook lets you scrub a video and click corners. ~40 frames at ~30-60 s each (including frame navigation) ≈ 30-45 minutes total.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.