ProjectSidewalk / ProjectSidewalk/RampNet

Mask the nadir/hood region in the reprojection rig; measure ΔAP

Open
#47 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
7
Forks
1
Avg merge
4d 11h
Merged PRs (30d)
7

Description

Motivation

equirect_tiling.default_views() uses pitch_deg=-30, so the bottom ~40% of every perspective view is the capture vehicle's hood and the black nadir cap — pixels that cannot contain a curb ramp. Roughly a third of every paid VLM call is spent there. It's not just wasteful:

  • Grounding DINO's highest-scoring box in a view is the hood itself (0.40), outranking its correct 0.22 box on a real tactile pad. Because AP ranks by score, hood detections at the top of the ranking depress AP directly — a plausible part of why GDINO's AP (0.032) trails OWLv2's (0.104) at similar operating points.

Change

Mask/crop the nadir + hood band before the model call (or drop detections whose center falls inside it). The band is a function of the rig geometry (pitch_deg, fov_v_deg), so it can be computed from the view, not hand-tuned per city. Consider raising/adjusting the pitch as an alternative to masking, and compare.

Now measurable

This was argued in docs/model_comparison.md (Next increments #1); the open-detector scores make its benefit quantifiable. Report ΔAP and ΔF1 vs the current rig, per model, on both cities, and perspective vs --tiling none side by side. This is also a fairness fix — the challengers are currently charged false positives for a fixed vehicle they can't avoid seeing, and the taxonomy pass (companion issue) will show how many FPs it accounts for.

Refs #39, #43.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with equirect_tiling.default_views() and the current reprojection rig, then read docs/model_comparison.md and inspect the perspective versus --tiling none evaluation paths. Compare masking, dropping detections, and pitch adjustment without assuming a hand-tuned band. Done means reporting ΔAP and ΔF1 versus the current rig for each model, on both cities and for both tiling modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.