ProjectSidewalk / ProjectSidewalk/RampNet
Train a supervised YOLO baseline (YOLO11 / YOLO26) on the RampNet dataset — isolate architecture vs. data
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 1
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 7
Description
Motivation
The comparison harness shows RampNet ≫ every zero-shot method (OWLv2, Grounding DINO, Molmo, Gemini, Qwen). It does not yet show RampNet > a generic supervised detector trained on the same auto-generated dataset. That's the most obvious reviewer challenge:
Is the ConvNeXt keypoint-heatmap architecture doing the work, or would any supervised detector trained on the RampNet dataset also beat the zero-shot field?
A trained YOLO answers exactly that — an architecture-vs-data ablation, and arguably the single strongest baseline the comparison is currently missing.
What to run
Train YOLO11 (mature, stable) and/or YOLO26 (Ultralytics' Jan-2026 model, NMS-free / DFL-free — gives the "latest YOLO" headline) on dataset/{train,val,test}, evaluate on the benchmark cities, add a compare.py table row. Skip v12/v13 as baselines (documented training-instability caveats).
Design frictions to resolve
-
Labels are points, not boxes (per
CLAUDE.md). Two paths:- (a) synthesize a fixed-size box around each GT point for training, then evaluate by box-center → GT-point at the existing radius
0.022(simplest, keeps eval identical; box size is a hyperparameter); or - (b) YOLO pose/keypoint mode to predict points directly (closest to RampNet, less standard tooling).
Recommend (a) first.
- (a) synthesize a fixed-size box around each GT point for training, then evaluate by box-center → GT-point at the existing radius
-
Input handling. RampNet ingests full 2048×4096 panos; YOLO wants ~640–1280 tiles. Either reuse the existing reprojection rig (
scripts/model_comparison/equirect_tiling.py, keeps it comparable to how VLMs are fed) or train on native pano tiles (comparable to how RampNet is fed). Pick one, hold it fixed, document it — it's a confound. -
Scoring. Reduce YOLO boxes to center points → the existing
rampnet/detection_eval.pymatcher at0.022. YOLO emits calibrated scores, so it gets AP / PR curve /--sweepfor free like the open detectors. Note the known caveat: RampNet's own AP is truncated (dets extracted at a 0.5 peak, a lower bound not comparable to detectors cached at 0.05), so lead with F1 at operating points and treat AP with that caveat.
Deliverable
A yolo provider path in compare.py, richmond + bend rows, and a short note on the train setup (box size, tiling choice, epochs).
Relates to #20 (harness for newer detection models/backbones). Companion zero-shot experiment: #52.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CLAUDE.md, compare.py, scripts/model_comparison/equirect_tiling.py, and rampnet/detection_eval.py. Resolve the box-versus-keypoint and tiling choices, train YOLO11 or YOLO26 on dataset/{train,val,test}, and reduce detections to points for evaluation at radius 0.022. Done means a yolo provider, richmond and bend rows, and a short note documenting box size, tiling, and epochs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100