Guide: four findings from the first clean-room `--env` regeneration

Observed on 2026-09-14, generating the Qdrant Edge course spec. A subagent with no memory of the authoring session ran /generate-spec --env=coding-agent-lab from the base guide (8e44ecb) and the new overlay (b0d88d6) alone, forbidden from reading any prior output. It produced an 836-line spec with 7 Decision Ledger rows. The hand-written file it was compared against has 4. Three rows agree; five subjects diverge.

Three defects belonged to the overlay and are fixed in 58ecfb9. The four below belong to the base guide and are not applied — they need a promotion pass with a validated regen property, per fix-and-test-travel-together.

F1 — The surfacing bar does not force one answer (the headline finding)

Two runs, same guide, same materials, opposite Ledgers on three subjects:

Subject	Clean-room run	Hand-written
Recognition threshold (0.80)	design-argued row	body default + Ask-First lever
Freshness re-ranking	baked in as a business rule	design-structural row
Photo-lane fetch depth	contradicted row	body default via the lever rule

The clean-room run flagged both of its calls as borderline and cited the guide text pulling each way. On the threshold it quoted §12.9's counter-example ("sensitivity alone does NOT qualify") against L5's narrated two-sided trade-off, and chose design-argued because the course hands the choice to the learner. On freshness it noted the design-argued bake-in litmus "points both ways" — meaning-only ranking is both a working stepping stone (L3/L4) and explicitly demonstrated failing (L5).

Implication. §13's Regeneration Test is failing on its own terms: "the built project changes between no-intake runs → the §3 default is ambiguous". Here it is the Ledger's composition, not a default, that moves. Every downstream artifact — the §0 gate's question count, resolved-decisions.md, both evals — is keyed to rows.

No rule proposed yet — logged for a think. The honest options look like (a) tighten the design-argued two-sided litmus so "a baseline the course moves beyond" and "an anti-pattern the course warns against" separate mechanically; (b) add a tie-break that prefers the business rule when both readings survive; (c) accept N-run variance on borderline subjects and require the generator to record which litmus branch it took, so at least the choice is auditable.

F2 — No category for a fixture-inexpressible input

§3 defines live as excluded-from-the-offline-run, for criteria needing network. The clean-room run hit an input a spec cannot author at all: a spoken audio recording. Synthetic silence produces no transcript, so the voice lane's acceptance criterion cannot run on authored fixtures.

It extended the mechanism itself, under a distinct tag, and said so.

Implication. Every course with an audio, video, or sensor-capture modality hits this. The guide's fixture doctrine ("facts you author") has no answer, so each generation invents one.

Proposed rule (§3, beside the live bar) — not applied:

A criterion whose input cannot be authored as a fixture (recorded speech, camera frames, sensor traces) is tagged live-<kind> and excluded from the offline run, exactly as live is. The spec must additionally carry an offline criterion covering the wiring that input feeds — the path is testable even when the payload is not.

F3 — Row-grouping order is ambiguous

§5.5 says the table is grouped learner → design (design-argued, design-structural) → realization → contradicted "with pattern-pipeline order within each group". Unclear whether design is one group (argued and structural interleaved by pipeline position) or two sub-blocks.

The clean-room run chose two sub-blocks as "the more literal reading of the parenthetical" and noted either reading produces a different table.

Proposed fix (§5.5 row order) — not applied: state it explicitly, either way. One sentence closes it.

F4 — §5 and §14 contradict each other on version pinning

• §5 step 5: "Decide the stack: pin modern versions as this build's choice"
• §14: "Version claims match the notebooks' actual install lines; era stated honestly if unpinned"

This course pins every dependency exactly. "Modern" and "matches the notebooks" are the same only by accident, and diverge as soon as a course ages.

Proposed fix — not applied: §5 concedes precedence to §14 where the course itself pinned. Suggested wording: pin the course's own versions when it pinned them, recording the pins as provenance; choose modern versions only where the course left them unpinned; either way §14's honesty requirement governs the claim.

---

Evidence: clean-room output at /tmp/cleanroom-spec.coding-agent-lab.md (836 lines, 2026-09-14). Overlay defects from the same run fixed in 58ecfb9. Slides-glob finding routed out to its own issue — it is skill tooling, not guide territory.

🤖 Generated with Claude Code

Inizia con le sezioni §3, §5.5, §5 passaggio 5, §12.9, §13 e §14 della guida di base, quindi riproduci /generate-spec --env=coding-agent-lab usando le revisioni di base e overlay citate. Confronta il Ledger rigenerato e gli artefatti downstream con le evidenze del clean-room. Il lavoro è completo quando le quattro issue della guida hanno risoluzioni esplicite, le modifiche proposte sono state applicate e la proprietà di rigenerazione convalida un output stabile.