basecamp / basecamp/basecamp-sdk

CI cannot verify spec/bc3-routes.json was regenerated from the pinned bc3 docs

Open
#589 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
49
Forks
12
Avg merge
20h 47m
Merged PRs (30d)
89

Description

Raised in review of #586. `make bc3-route-parity` runs offline against the vendored `spec/bc3-routes.json`, which is what makes it unskippable — but it also means CI never re-derives the table from bc3.

## What already holds offline

`bc3-route-parity` fails when:

1. the table's `source.revision` != `spec/api-provenance.json`'s `bc3.revision`;
2. the table's `source.generator_sha256` != a SHA-256 of `scripts/generate-bc3-routes` + `scripts/bc3_route_normalizer.rb` — so a changed extractor with a stale table fails, which was the larger half of the concern;
3. the extracted route count falls below a pinned floor (a broken regex can't yield an empty set that passes vacuously).

## The residual hole

**A hand-edited `source.revision` that matches the pin without regeneration.** Nothing offline can detect that — the table would claim the right pin, carry the right generator fingerprint, and contain stale routes. Detecting it requires reading bc3.

`make bc3-routes-check` does exactly that (regenerate at the pin into a `mktemp -d`, diff), but it needs `BC3_REPO_PATH` and **no workflow checks out bc3 today** — there is no existing pattern for it anywhere in `.github/workflows/`. Provisioning a bc3 access secret is a decision beyond that PR's scope, so the Makefile comment now states the limitation plainly instead of claiming a nightly run that does not exist. (It previously claimed one. It was wrong.)

## What closing this looks like

A scheduled job that:

1. sparse-checks out `basecamp/bc3` (`doc/api/sections`, `config/routes.rb`) at the revision in `spec/api-provenance.json`, using a repo secret;
2. runs `make bc3-routes-check`;
3. **hard-fails when the secret is missing.** Not a graceful skip — that is the same silent-no-op failure mode as `live-canary.yml:71-84`, which exits 0 on every trigger including the nightly cron. A freshness gate that can't tell "fresh" from "couldn't look" is not a gate.

Practical note: the pinned revision must be reachable from the checkout. A shallow clone of `master` will not contain it once the pin ages, so fetch the specific SHA or use sufficient depth.

Lower priority than it looks: the repin workflow is deliberate and human-driven, and `AGENTS.md`'s pin rule plus `bc3-route-parity`'s direction-2 ledger already force per-repin triage. This closes the last "table lies about its own provenance" gap.

Contributor guide

Open the contributing guide

Research direction

Start with Makefile's bc3-routes-check, spec/api-provenance.json, and the existing .github/workflows/ directory, comparing the failure behavior in live-canary.yml:71-84. Check how a workflow can fetch basecamp/bc3 at the pinned revision and require the repository secret. Done means a scheduled job runs the check against that revision and fails when the secret is missing or the regenerated table differs.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github-actions
Domain
ci-cd, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.