ProjectSidewalk / ProjectSidewalk/RampNet

360 pano seam: curb ramps split across the left/right edge get double-counted in GT scoring

Open
#43 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation enhancement
Dominant language
Python
Stars
7
Forks
1
Avg merge
4d 11h
Merged PRs (30d)
7

Description

Problem

A 360° equirectangular pano has a hard seam at its left/right edge (x≈0 and x≈1 are the
same physical location — the pano wraps there). A curb ramp that straddles that seam gets
bisected: one half at the far right, one at the far left. This creates a ground-truth
scoring hazard that the current verdict schema and viewer don't guard against.

First real case — Clovis 150809456962552 (captured 2019-07), benchmark/clovis:

  • The model emitted det1 at x=0.9883, y=0.586, conf 0.580 — the right-hand half of a
    curb ramp on the corner directly behind the car.
  • Reviewing the left edge (x≈0.003), the same corner's other half looks like an
    undetected ramp, so the natural move is to drop a missed mark there.
  • Result: the same physical ramp is scored as both a true positive (det1 correct)
    and a false negative (the missed mark). Incoherent, and it docks recall for a ramp
    the model actually found. (The other detection on this pano, det0 at x=0.3545, and a
    separate genuine miss at x≈0.28, are unaffected — those aren't near the seam.)

Why it's easy to miss

The GT viewer (scripts/gt_gallery.py) treats the pano as a flat, hard-edged image:
clampPan() clamps horizontal pan to [w − w·zoom, 0], so you cannot pan across the
seam
. The two halves appear at opposite ends of the strip with no visual cue they're the
same corner. The model sees the same hard seam (no wrap-padding on its 4096×2048 input), so
this is a shared artifact of how the pano is cut, not just an annotation slip.

Scope (what this is and isn't)

  • GT scoring — the real issue. The schema auto-dedups redundant detections (the
    duplicate verdict), but there is no dedup between a detection and a missed mark
    (different categories), so the seam case relies entirely on reviewer discipline.
  • Production / PS submission — fine, no action. A single detection at x≈0.99 submits
    normally, and even a both-halves double-detection projects to ~the same geographic point,
    so PS clustering absorbs it (same as any duplicate label).
  • Model limitation — separate thread. Seam-straddling ramps are genuinely harder for the
    model (no horizontal wrap). Worth tracking as a model/pipeline question (wrap-pad the input,
    or detect on a seam-centered rotation and merge), but out of scope here.

Reviewer rule (mentally wrap the seam)

When a ramp straddles x≈0 / x≈1:

  1. Model hit either side → one TP. Do not add a missed mark on the other side.
  2. Model hit both sides → the second is a duplicate (FP by default) — the seam just
    spreads the redundant pair to opposite edges.
  3. Model hit neither side → one missed mark, placed once.

Proposed changes

  • Document the reviewer rule in the viewer help text and benchmark/README.md.
  • Viewer (optional, cheap): either wrap-pad the pan strip so you can scroll across
    the seam continuously, or a toggle that re-centers the view on the seam (180° rotation)
    for a second scan.
  • Scorer backstop (optional): warn when a missed mark near one seam edge pairs with
    a detection near the opposite edge at a similar y — catches the double-count even if a
    reviewer forgets the rule.

Immediate action

Clovis 150809456962552: keep det1 correct (one TP), drop the left-seam missed mark. Applied
during the in-progress Clovis GT review. Only seam case seen so far, but the Clovis AOI is a
broad ~12×13 km box, so more are plausible.

Related: #26 (GT tool port + UX/scoring), #21 (benchmark dataset).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect scripts/gt_gallery.py and benchmark/README.md, then trace the GT verdict and scoring path described in the issue. Reproduce the Clovis 150809456962552 seam case and determine which proposed viewer or scorer safeguard is accepted. Done means the reviewer rule is documented and the chosen safeguard prevents a single seam-spanning ramp from being counted as both a true positive and a missed mark.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.