hans / hans/dot-prediction

Phase 1c: add big_star as 5th anchor in per-frame H fit (eliminate TR/TL extrapolation)

Open
#8 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Follow-up to #6 (PR #7).

## Motivation

Today's Phase 1c per-frame H is fit from 4 correspondences (`screen_bl`, `screen_br`, `box_bl`, `box_br`) that are all clustered near the screen bottom + photodiode device. Screen TR/TL are far from any anchor → high-leverage extrapolation. Sub-pixel anchor jitter → thousands of px of TR jitter; bimodal detector regimes → visible polygon jumps. PR #5 (H-element smoothing) and PR #7 (anchor-refit smoothing) both attack this with temporal smoothing, but neither addresses the root cause: the anchor geometry itself is ill-conditioned for projecting TR/TL.

A 5th anchor in the middle of the iPad screen would dramatically change the conditioning. The big_star (freshly-revealed dot in the experiment) is detectable per-frame and we know exactly where on the iPad screen it lives (from `trials_with_video.parquet` + `behavior_to_screen()`).

big_star is currently used **only for held-out validation** in `homography_solver.big_star_residuals()` and the Phase-1c eval scatter/hist. This issue proposes promoting it to a fit anchor.

## What 'big_star in fit' means concretely

Phase 1c is the right place — it's where the per-frame fit happens (`notebooks/phase1c_eval.py` + `src/homography_solver.py`). NOT Phase 2 (that's pre-projection cleanup, where smoothing belongs).

Per-frame work:

1. **Active dot lookup.** For each video frame, find the most-recently-revealed trial point via `trials_with_video.parquet` (already done in `big_star_residuals()`). Compute the true screen-px position via `homography_solver.behavior_to_screen()`.

2. **Per-frame big_star detection.** Two options:
- **Local search around predicted position.** Project the active dot's screen-xy through the prior frame's H (forward pass) to predict its frame-px location. Run `src/local_star_detector.py` to confirm/refine. Cheap, high precision, but tracking dependency.
- **Global blob detection.** Run `src/star_detector.py`'s warm-blob detector per frame; pick the brightest blob inside the screen polygon. More robust to tracking loss, slower per frame.

3. **5-point H fit.** Modify `homography_solver.fit_per_frame_homography()` (and `phase1c_eval`'s `_process_frame`) to accept an optional 5th correspondence. `cv2.findHomography` accepts ≥4 points; 5-point case overdetermines the 8-DOF model and improves conditioning across the iPad screen rectangle. Existing geometric/temporal-jump checks should still apply.

4. **Re-validation.** The current held-out validation (big_star residuals on labeled frames) becomes moot — big_star is now in the fit. Replace it with leave-one-out cross-validation OR introduce a different held-out signal (the small stars from the local detector are candidates).

## Acceptance criterion

- Polygon overlay (especially TR/TL corners) is tight across the full video, not just smoothed clips. Whole-video TR_x catastrophic outliers (currently O(1M px) at a handful of frames) should disappear.
- Click→canvas validation median ideally well under 250 canvas-px (current 322; the gate is < 250).
- No regression on Phase 1c eval metrics that aren't directly entangled with big_star (homography_valid rate, on_screen rate).

## Out of scope for this issue

- The temporal smoothing in Phase 2 (#7) stays — anchor-refit + 5-anchor fit are complementary. With big_star in the fit, the bad-regime amplitude shrinks so the smoothing window can probably also shrink.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with notebooks/phase1c_eval.py's _process_frame and src/homography_solver.py, including fit_per_frame_homography(), big_star_residuals(), and behavior_to_screen(); review the existing detectors in src/local_star_detector.py and src/star_detector.py. Add the per-frame big_star correspondence and 5-point fit, then replace the current held-out validation and check polygon overlays, click→canvas error, homography validity, and on-screen rate against the acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
opencv, python
Domain
computer-vision
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.