populace-data load() bypasses the release certificate: mutable root fetch with no revision, SHA, or engine verification

Summary

populace.data.load() — the normal consumption path — bypasses everything the release machinery certifies. It fetches the mutable root H5 with no revision pin, never reads latest.json, never verifies the artifact SHA against the release manifest, and never checks the engine version against the certified compatibility specifier. Publication ordering makes this worse: root artifacts are overwritten first, in separate commits, before the release files, tag, and pointer — so a failed or in-flight publication exposes an uncertified artifact to every ordinary consumer while latest.json still points at the previous release.

Evidence (current main, 8264c49)

• packages/populace-data/src/populace/data/loader.py:96-124 — download() calls hf_hub_download(repo_id, filename, repo_type="dataset") with no revision: it fetches whatever the repo root currently holds.
• loader.py:127-163 — load() resolves the registry spec, imports whichever compatible engine is installed, and constructs the dataset from download(...). It reads none of: latest.json, release_manifest.json, artifact sha256, compatibility specifiers.
• packages/populace-data/pyproject.toml:27 — the extra is policyengine-us>=1.745.0,<2; the certified exact engine version (recorded with == in the release manifest, verified by contract.py:626-697 at publish time) is never enforced at load time.
• packages/populace-data/src/populace/data/release.py:209-243 — upload order is: root artifacts (each its own commit) → releases/<id>/ files → tag → latest.json pointer last. The pointer-last ordering protects only consumers that resolve the pointer — which load() is not.

Consequences (all mechanical, none hypothetical):

1. A publication that fails after the root-H5 upload leaves the new uncertified H5 as what load() returns, with the old pointer current and the new release evidence incomplete.
2. Any consumer environment resolves a potentially different engine version than the one the artifact was certified against — and cross-release comparisons can silently mix engine versions.
3. There is no way to load a specific certified release through the public API at all (no revision=release_id path).

Fix sketch (release-native loading)

1. load() / download() resolve latest.json (or an explicit release_id= argument), fetch releases/<id>/release_manifest.json, download artifacts at revision=<release tag/commit>, verify SHA-256 against the manifest, and check the installed engine against the manifest's compatibility specifier (hard error or loud warning — decide once).
2. Publish immutable release files before overwriting root aliases; ideally upload the root alias last (it is a convenience alias, not the source of truth), or in a single commit with the pointer.
3. Keep the mutable-root path only as an explicitly named legacy escape hatch, not the default.

Acceptance criteria

• load("us", 2024) verifies artifact SHA-256 against the resolved release manifest before constructing the dataset.
• load(..., release_id=...) (or equivalent) can fetch a pinned historical release at an immutable revision.
• Engine-compatibility mismatch at load time is surfaced (error or warning) using the manifest's recorded specifier.
• Publication uploads release files and pointer such that no ordinary consumer can observe a new root artifact before its certificate is complete.
• A test simulates a publication failure between root upload and pointer flip and asserts consumers cannot silently receive the uncertified artifact.

Cross-links

• #366 (release-delta report — the consumer adoption contract this loader gap undermines)
• #204 (us-data replacement: downstream consumers are being pointed at this loader)
• #224 / #213 (published variants — same loader contract)
• #305 (validation portfolio META)

Surfaced by an external architecture review (Codex/gpt-5.6-sol, 2026-07-09); citations independently verified.

Read packages/populace-data/src/populace/data/loader.py:96-163 and release.py:209-243, then inspect contract.py:626-697 and the engine extra in pyproject.toml:27. Trace how latest.json, release manifests, revisions, hashes, and compatibility are currently handled. Done means pinned release loading, manifest verification, engine compatibility handling, safe publication ordering, and coverage for failure between root upload and pointer flip.