ProjectSidewalk / ProjectSidewalk/RampNet

model_comparison.md and curb_ramp_data_sourcing.md are too long to be read whole: split the runbooks out and generate the tables, don't trim prose

Open
#145 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Python
Stars
7
Forks
1
Avg merge
4d 11h
Merged PRs (30d)
7

Description

Two docs have grown past the point where anyone — human or agent — reliably reads them whole.
Measured on main at 0cba7b2:

doc lines bytes ~tokens
docs/curb_ramp_data_sourcing.md 2,450 160 KB ~40k
docs/model_comparison.md 2,021 136 KB ~34k
docs/operating_point.md 813 51 KB ~13k
docs/replication.md 748 49 KB ~12k

The failure mode is specific and it is already happening: an agent that cannot hold the whole
file re-derives a number that is already in it, or contradicts a caveat it never read.
The
WITNESS_POOL_46 and y11x_tiles incidents were both this. A caveat that is not read is
functionally the same as a caveat that was never written, which defeats the point of the
standing rule that caveats travel with their numbers.

The fix is structural, not editorial

Trimming prose is the wrong instinct here and I want to argue against it explicitly. These
files are long because they carry real content — negative results, caveats, and run instructions
that the repo's standing rules require to exist. Deleting sentences to hit a line count trades a
readability problem for a replication problem, and risks orphaning a caveat from its number,
which is the one thing CLAUDE.md says must not happen.

docs/model_comparison.md is really three documents fused, and they have different readers and
different lifetimes:

  1. Results tables — largely superseded by docs/model_scoreboard.md (#141), which is
    generated from committed data and has six <!-- BEGIN GENERATED --> blocks plus a
    scoreboard.py --check. model_comparison.md has zero generated blocks; every table in
    it is hand-maintained prose that can drift silently.
  2. Methodology and validation — reprojection, box-coordinate mapping, per-provider quirks,
    null-recall, FP taxonomy. This is the load-bearing part and the reason to open the file.
  3. Runbooks — credentials, "Running it", the Hyak launcher walkthrough. ~370 lines from
    § "Running it" to the end. Operational, read while doing rather than while thinking.

Splitting (3) into docs/running_model_comparison.md and letting the generated scoreboard own
(1) removes roughly 40% of the file without deleting a sentence of content.

Proposed scope

  • Move § "Running it", § "Running the open-weight models on Hyak", and § "Gemini credentials"
    into docs/running_model_comparison.md; leave a one-line pointer
  • Replace hand-maintained result tables with <!-- BEGIN GENERATED --> blocks fed by
    scoreboard.py, extending the #141 pattern, so tables cannot drift
  • Keep every caveat physically adjacent to the number it qualifies — this is the constraint
    the split has to respect, and it is worth a reviewer checking explicitly
  • Add a short "start here" map at the top of each remaining long doc
  • Same treatment for docs/curb_ramp_data_sourcing.md, which is larger and has the same
    shape (findings + per-city runbooks)

Explicitly not in scope

  • Deleting negative results, caveats, or methodology to save space
  • Any change that separates a number from the caveat that qualifies it
  • Rewriting for tone

Done when

  • No doc in docs/ exceeds ~1,200 lines without a generated-block or split rationale stated
    in the file
  • model_comparison.md's tables are generated and --check covers them
  • A reader landing cold can tell within 20 lines which file answers their question

Related: #141 (the generated-scoreboard pattern to extend), #122.


🤖 Generated with Claude Code (claude-opus-5[1m])

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading docs/model_comparison.md, docs/curb_ramp_data_sourcing.md, related issue #141, and the referenced scoreboard.py --check pattern. Identify the runbook sections and hand-maintained tables, then preserve caveat-number adjacency while splitting and generating content. Done means the stated line-count rationale, generated blocks, check coverage, and start-here maps are in place.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.