ProjectSidewalk / ProjectSidewalk/RampNet
Mask the nadir/hood region in the reprojection rig; measure ΔAP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 1
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 7
Description
Motivation
equirect_tiling.default_views() uses pitch_deg=-30, so the bottom ~40% of every perspective view is the capture vehicle's hood and the black nadir cap — pixels that cannot contain a curb ramp. Roughly a third of every paid VLM call is spent there. It's not just wasteful:
- Grounding DINO's highest-scoring box in a view is the hood itself (0.40), outranking its correct 0.22 box on a real tactile pad. Because AP ranks by score, hood detections at the top of the ranking depress AP directly — a plausible part of why GDINO's AP (0.032) trails OWLv2's (0.104) at similar operating points.
Change
Mask/crop the nadir + hood band before the model call (or drop detections whose center falls inside it). The band is a function of the rig geometry (pitch_deg, fov_v_deg), so it can be computed from the view, not hand-tuned per city. Consider raising/adjusting the pitch as an alternative to masking, and compare.
Now measurable
This was argued in docs/model_comparison.md (Next increments #1); the open-detector scores make its benefit quantifiable. Report ΔAP and ΔF1 vs the current rig, per model, on both cities, and perspective vs --tiling none side by side. This is also a fairness fix — the challengers are currently charged false positives for a fixed vehicle they can't avoid seeing, and the taxonomy pass (companion issue) will show how many FPs it accounts for.
Refs #39, #43.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with equirect_tiling.default_views() and the current reprojection rig, then read docs/model_comparison.md and inspect the perspective versus --tiling none evaluation paths. Compare masking, dropping detections, and pitch adjustment without assuming a hand-tuned band. Done means reporting ΔAP and ΔF1 versus the current rig for each model, on both cities and for both tiling modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100