ProjectSidewalk / ProjectSidewalk/RampNet
model_comparison.md and curb_ramp_data_sourcing.md are too long to be read whole: split the runbooks out and generate the tables, don't trim prose
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 1
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 7
Description
Two docs have grown past the point where anyone — human or agent — reliably reads them whole.
Measured on main at 0cba7b2:
| doc | lines | bytes | ~tokens |
|---|---|---|---|
docs/curb_ramp_data_sourcing.md |
2,450 | 160 KB | ~40k |
docs/model_comparison.md |
2,021 | 136 KB | ~34k |
docs/operating_point.md |
813 | 51 KB | ~13k |
docs/replication.md |
748 | 49 KB | ~12k |
The failure mode is specific and it is already happening: an agent that cannot hold the whole
file re-derives a number that is already in it, or contradicts a caveat it never read. The
WITNESS_POOL_46 and y11x_tiles incidents were both this. A caveat that is not read is
functionally the same as a caveat that was never written, which defeats the point of the
standing rule that caveats travel with their numbers.
The fix is structural, not editorial
Trimming prose is the wrong instinct here and I want to argue against it explicitly. These
files are long because they carry real content — negative results, caveats, and run instructions
that the repo's standing rules require to exist. Deleting sentences to hit a line count trades a
readability problem for a replication problem, and risks orphaning a caveat from its number,
which is the one thing CLAUDE.md says must not happen.
docs/model_comparison.md is really three documents fused, and they have different readers and
different lifetimes:
- Results tables — largely superseded by
docs/model_scoreboard.md(#141), which is
generated from committed data and has six<!-- BEGIN GENERATED -->blocks plus a
scoreboard.py --check.model_comparison.mdhas zero generated blocks; every table in
it is hand-maintained prose that can drift silently. - Methodology and validation — reprojection, box-coordinate mapping, per-provider quirks,
null-recall, FP taxonomy. This is the load-bearing part and the reason to open the file. - Runbooks — credentials, "Running it", the Hyak launcher walkthrough. ~370 lines from
§ "Running it" to the end. Operational, read while doing rather than while thinking.
Splitting (3) into docs/running_model_comparison.md and letting the generated scoreboard own
(1) removes roughly 40% of the file without deleting a sentence of content.
Proposed scope
- Move § "Running it", § "Running the open-weight models on Hyak", and § "Gemini credentials"
intodocs/running_model_comparison.md; leave a one-line pointer - Replace hand-maintained result tables with
<!-- BEGIN GENERATED -->blocks fed by
scoreboard.py, extending the #141 pattern, so tables cannot drift - Keep every caveat physically adjacent to the number it qualifies — this is the constraint
the split has to respect, and it is worth a reviewer checking explicitly - Add a short "start here" map at the top of each remaining long doc
- Same treatment for
docs/curb_ramp_data_sourcing.md, which is larger and has the same
shape (findings + per-city runbooks)
Explicitly not in scope
- Deleting negative results, caveats, or methodology to save space
- Any change that separates a number from the caveat that qualifies it
- Rewriting for tone
Done when
- No doc in
docs/exceeds ~1,200 lines without a generated-block or split rationale stated
in the file -
model_comparison.md's tables are generated and--checkcovers them - A reader landing cold can tell within 20 lines which file answers their question
Related: #141 (the generated-scoreboard pattern to extend), #122.
🤖 Generated with Claude Code (claude-opus-5[1m])
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading docs/model_comparison.md, docs/curb_ramp_data_sourcing.md, related issue #141, and the referenced scoreboard.py --check pattern. Identify the runbook sections and hand-maintained tables, then preserve caveat-number adjacency while splitting and generating content. Done means the stated line-count rationale, generated blocks, check coverage, and start-here maps are in place.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, tooling
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100