ProjectSidewalk / ProjectSidewalk/RampNet
Re-run and publish the Vistas 1024-parity and A40-control detections (#126, #137)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 1
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 7
Description
The Vistas resolution-parity result in #137 (docs/model_comparison.md, the 1024-input parity arm and the same-env 384 A40 control) rests on detections that no longer exist anywhere. The only copy was a private .model_cache shard in a scratch worktree on makelab2; a search on 2026-09-17 found no registered worktree under /homes/gws/jonf/RampNet, no shard written 2026-08-17..20, and none of the parity run's shard names (compare.cache_key over the 1024 signature) under /homes/gws/jonf, /tmp, /var, the root filesystem or the lab mounts. The doc states this beside the numbers; #137 was merged with the gap stated rather than held.
The fix is cheap on GPU — both A40 arms re-run in about 3m38s on makelab2 — but publishing the result into benchmark/model_detections/ runs into three things the roster's own rules leave open, and they need a decision before the re-run PR:
- A pinned 1024 leg forces the published 384 file to be renamed too (
test_every_leg_of_a_pinned_model_is_qualified), and the pin-naming rule expects a scalar. - The 384 control shares the published run's cache key by design, so it cannot be a pinned leg as the roster stands.
scripts/analysis/export_model_cache.pydoes not take--vistas-input-size.
Plan: settle 1–3 in rampnet/roster.py / the exporter, re-run both arms on makelab2, publish, and update the five doc sections that quote the parity numbers. The two tests that pin the pre-fix cascade state on purpose (tests/test_cascade_gate.py) will need the same treatment if the re-run does not reproduce the old numbers exactly — and if it does not, that is a result to record, not a discrepancy to smooth over.
🤖 Generated with Claude Code — Opus 5 (1M context), claude-opus-5[1m]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading rampnet/roster.py and scripts/analysis/export_model_cache.py, then inspect the relevant tests in tests/test_cascade_gate.py and the roster qualification tests. Run the existing tests before deciding how the pinned 1024 leg, 384 control, and input-size export should be handled. Done means both A40 arms are rerun, detections are published under benchmark/model_detections/, and the five documented parity sections reflect the recorded results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, documentation, machine-learning, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100