Extend RampNet output from points to boxes/polygons (point-prompted SAM2; CenterNet-style size head; segmentation)

Context

RampNet's output is points — by design, because Stage 1 translates government GPS locations into pixel points, and the model regresses fixed-sigma Gaussians (sigma 10 on the 512×1024 heatmap) around them. Extent (bounding box / polygon) is nowhere in the supervision, so the predicted blob size reflects target sigma and confidence, not ramp size. Thresholding the current heatmaps cannot recover object extent, and nothing in the 850k-label training set can teach it.

But extent would be valuable: it upgrades the output from "where" to "where and how big," enables area/geometry-based condition features (RampNet-2.0-adjacent), and makes RampNet directly comparable to box-native baselines (#51) on their own terms.

Proposed paths (increasing effort)

1. Point-prompted SAM2 — boxes and polygons with zero training

RampNet's high-precision points (P 0.949 at the 0.55 operating point) are exactly what SAM-family models accept as prompts:

• RampNet detects a point on the pano
• crop a local region around it at full pano resolution (prompting on a crop sidesteps equirectangular distortion and the small-object problem)
• SAM2 with the point prompt → instance mask → polygon; tight bbox for free

RampNet stays the detector; SAM adds extent. This also doubles as a label-minting machine: run it over the ~850k Stage 1 training points to generate pseudo-boxes/masks at scale for paths 2–3.

Main risk: segmentation quality on low-contrast ramps that blend into surrounding concrete. This is an empirical question, and we can answer it cheaply — see "First experiment" below.

2. CenterNet-style size head on the existing architecture

Keep the ConvNeXt V2 backbone and center heatmap; add two regression channels (w, h at the peak). Minimal delta to native boxes, preserves the one-pass story. Needs box supervision at scale — which path 1 mints (SAM pseudo-boxes, filtered by mask quality). The 1k gold boxes are too few to train on and are the benchmark; they stay eval-only.

3. Full segmentation output

If polygons become first-class: Mapillary Vistas includes a curb-cut class with polygon annotations (coverage/quality to be verified before committing) — a fine-tune source that dovetails with the Mapillary direction (#25, #48) and sidewalk-auto-labeler. SAM pseudo-masks scale that supervision the same way Stage 1 scaled points.

First experiment (cheap, decisive)

manual_labels/ is YOLO-format boxes — 3,919 of them on the 1,000 gold panos — which we have only ever consumed as centers. Run point→SAM2 on the gold panos (prompting from GT centers to isolate segmentation quality from detection quality, then from RampNet detections for the end-to-end number) and report the IoU distribution against the gold boxes. One day of GPU time; the IoU histogram tells us whether path 1 is production-grade or needs work, and is a committable result either way (per the repo's negative-results rule).

Related

• #51 / #70 — supervised YOLO baselines (box-native comparison this would enable on equal terms)
• #35 — hybrid RampNet + foundation-model cascades (same "RampNet as prompter" shape)
• #48 — multi-view fusion / RampNet 2.0 (extent + geometry feeds correspondence)
• #25 — Mapillary high-res experiments (path 3 supervision source)
• #20 — newer-model harness (where a size-head variant would be trained/evaluated)

🤖 Generated with Claude Code (claude-fable-5)

Start with the YOLO-format boxes in manual_labels/ and the 1,000 gold panos. Run the point→SAM2 experiment first using ground-truth centers, then RampNet detections, and compare mask-derived boxes with the gold boxes using an IoU distribution. Done means reporting the histogram and using it to decide whether the SAM2 path is viable before pursuing a size head or full segmentation.