Phase 1c: add big_star as 5th anchor in per-frame H fit (eliminate TR/TL extrapolation)
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Follow-up to #6 (PR #7).
## Motivation
Today's Phase 1c per-frame H is fit from 4 correspondences (`screen_bl`, `screen_br`, `box_bl`, `box_br`) that are all clustered near the screen bottom + photodiode device. Screen TR/TL are far from any anchor → high-leverage extrapolation. Sub-pixel anchor jitter → thousands of px of TR jitter; bimodal detector regimes → visible polygon jumps. PR #5 (H-element smoothing) and PR #7 (anchor-refit smoothing) both attack this with temporal smoothing, but neither addresses the root cause: the anchor geometry itself is ill-conditioned for projecting TR/TL.
A 5th anchor in the middle of the iPad screen would dramatically change the conditioning. The big_star (freshly-revealed dot in the experiment) is detectable per-frame and we know exactly where on the iPad screen it lives (from `trials_with_video.parquet` + `behavior_to_screen()`).
big_star is currently used **only for held-out validation** in `homography_solver.big_star_residuals()` and the Phase-1c eval scatter/hist. This issue proposes promoting it to a fit anchor.
## What 'big_star in fit' means concretely
Phase 1c is the right place — it's where the per-frame fit happens (`notebooks/phase1c_eval.py` + `src/homography_solver.py`). NOT Phase 2 (that's pre-projection cleanup, where smoothing belongs).
Per-frame work:
1. **Active dot lookup.** For each video frame, find the most-recently-revealed trial point via `trials_with_video.parquet` (already done in `big_star_residuals()`). Compute the true screen-px position via `homography_solver.behavior_to_screen()`.
2. **Per-frame big_star detection.** Two options:
- **Local search around predicted position.** Project the active dot's screen-xy through the prior frame's H (forward pass) to predict its frame-px location. Run `src/local_star_detector.py` to confirm/refine. Cheap, high precision, but tracking dependency.
- **Global blob detection.** Run `src/star_detector.py`'s warm-blob detector per frame; pick the brightest blob inside the screen polygon. More robust to tracking loss, slower per frame.
3. **5-point H fit.** Modify `homography_solver.fit_per_frame_homography()` (and `phase1c_eval`'s `_process_frame`) to accept an optional 5th correspondence. `cv2.findHomography` accepts ≥4 points; 5-point case overdetermines the 8-DOF model and improves conditioning across the iPad screen rectangle. Existing geometric/temporal-jump checks should still apply.
4. **Re-validation.** The current held-out validation (big_star residuals on labeled frames) becomes moot — big_star is now in the fit. Replace it with leave-one-out cross-validation OR introduce a different held-out signal (the small stars from the local detector are candidates).
## Acceptance criterion
- Polygon overlay (especially TR/TL corners) is tight across the full video, not just smoothed clips. Whole-video TR_x catastrophic outliers (currently O(1M px) at a handful of frames) should disappear.
- Click→canvas validation median ideally well under 250 canvas-px (current 322; the gate is < 250).
- No regression on Phase 1c eval metrics that aren't directly entangled with big_star (homography_valid rate, on_screen rate).
## Out of scope for this issue
- The temporal smoothing in Phase 2 (#7) stays — anchor-refit + 5-anchor fit are complementary. With big_star in the fit, the bad-regime amplitude shrinks so the smoothing window can probably also shrink.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with notebooks/phase1c_eval.py's _process_frame and src/homography_solver.py, including fit_per_frame_homography(), big_star_residuals(), and behavior_to_screen(); review the existing detectors in src/local_star_detector.py and src/star_detector.py. Add the per-frame big_star correspondence and 5-point fit, then replace the current held-out validation and check polygon overlays, click→canvas error, homography validity, and on-screen rate against the acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- opencv, python
- Domain
- computer-vision
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100