ProjectSidewalk / ProjectSidewalk/RampNet

Train a supervised YOLO baseline (YOLO11 / YOLO26) on the RampNet dataset — isolate architecture vs. data

Open
#51 30 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
7
Forks
1
Avg merge
4d 11h
Merged PRs (30d)
7

Description

Motivation

The comparison harness shows RampNet ≫ every zero-shot method (OWLv2, Grounding DINO, Molmo, Gemini, Qwen). It does not yet show RampNet > a generic supervised detector trained on the same auto-generated dataset. That's the most obvious reviewer challenge:

Is the ConvNeXt keypoint-heatmap architecture doing the work, or would any supervised detector trained on the RampNet dataset also beat the zero-shot field?

A trained YOLO answers exactly that — an architecture-vs-data ablation, and arguably the single strongest baseline the comparison is currently missing.

What to run

Train YOLO11 (mature, stable) and/or YOLO26 (Ultralytics' Jan-2026 model, NMS-free / DFL-free — gives the "latest YOLO" headline) on dataset/{train,val,test}, evaluate on the benchmark cities, add a compare.py table row. Skip v12/v13 as baselines (documented training-instability caveats).

Design frictions to resolve

  • Labels are points, not boxes (per CLAUDE.md). Two paths:

    • (a) synthesize a fixed-size box around each GT point for training, then evaluate by box-center → GT-point at the existing radius 0.022 (simplest, keeps eval identical; box size is a hyperparameter); or
    • (b) YOLO pose/keypoint mode to predict points directly (closest to RampNet, less standard tooling).

    Recommend (a) first.

  • Input handling. RampNet ingests full 2048×4096 panos; YOLO wants ~640–1280 tiles. Either reuse the existing reprojection rig (scripts/model_comparison/equirect_tiling.py, keeps it comparable to how VLMs are fed) or train on native pano tiles (comparable to how RampNet is fed). Pick one, hold it fixed, document it — it's a confound.

  • Scoring. Reduce YOLO boxes to center points → the existing rampnet/detection_eval.py matcher at 0.022. YOLO emits calibrated scores, so it gets AP / PR curve / --sweep for free like the open detectors. Note the known caveat: RampNet's own AP is truncated (dets extracted at a 0.5 peak, a lower bound not comparable to detectors cached at 0.05), so lead with F1 at operating points and treat AP with that caveat.

Deliverable

A yolo provider path in compare.py, richmond + bend rows, and a short note on the train setup (box size, tiling choice, epochs).


Relates to #20 (harness for newer detection models/backbones). Companion zero-shot experiment: #52.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CLAUDE.md, compare.py, scripts/model_comparison/equirect_tiling.py, and rampnet/detection_eval.py. Resolve the box-versus-keypoint and tiling choices, train YOLO11 or YOLO26 on dataset/{train,val,test}, and reduce detections to points for evaluation at radius 0.022. Done means a yolo provider, richmond and bend rows, and a short note documenting box size, tiling, and epochs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.