Overlay: six gaps surfaced by the first two clean-room `--env=coding-agent-lab` runs

Observed 2026-09-14. Two clean-room subagents generated the Qdrant Edge spec through /generate-spec --env=coding-agent-lab, one against overlay b0d88d6 and one against 58ecfb9. Three defects they found are already fixed in 58ecfb9. These six are not applied — they need their own revision plus a regen that shows the fix landed.

Overlay files: .claude/skills/generate-spec/references/spec-generation-guide.coding-agent-lab.md and its .runtime.md companion.

G1 — The overlay never states the deliverable's shape (highest priority)

The second run reported this as its single biggest judgment call. Nothing in the base guide or the overlay says what artifact a lab build produces. The course's own artifact is a Jupyter notebook — which the lab has no surface for. The run inferred a served web app from runtime facts that only describe how the preview works if you serve something:

One server, port 4000, bound to 0.0.0.0, serving both the page and the API.
Every URL inside the page must be relative … belongs in the spec as a numbered business rule with an acceptance criterion, not as a note.

That language presupposes a page. environments/coding-agent-lab.md §1 and §10 do make a served app the environment's default shape — so the inference was right — but the overlay never says so, and a future run could infer a CLI instead with nothing to stop it. Currently harmless by luck.

Proposed fix — not applied: state the deliverable shape in the rules file, as a default with an escape: the lab's default artifact is a single server on port 4000 serving both a page and its API; a course whose pattern has no user-facing surface may deliver a library plus its test suite, and must say so in §1.

G2 — ADD 2 forbids a co-label that some constraints genuinely need

ADD 2: "It must never also carry a course-demonstrated label."

But "release the speech model before loading the embedders" is independently (a) demonstrated by the course's own helper, with a stated memory-budget reason, and (b) required by the runtime facts ("A spec whose pipeline can hold several models resident at once must carry a business rule releasing one before loading the next"). The run wrote it as "course-demonstrated, reinforced [environment]" — arguably the banned co-label — judging that splitting it into two near-duplicate rules was worse.

Proposed fix — not applied: ADD 2 gains a third form for a constraint both sources independently require, e.g. course-demonstrated, reinforced [environment]. The ban's real target is claiming course provenance for an environment-only fact; it should not forbid recording that both are true.

G3 — ADD 3's AC obligation has no branch for an empty Invariant

ADD 3: "Every ANSWERED or CONSTRAINED dimension's Invariant is exercised by ≥1 acceptance criterion at its new home, or the table states why it cannot be."

The project dimension's Invariant is empty by base §3's own rule ("typically empty for project/data/goal/scope-boundary"). There is nothing to exercise — which is not the same as "cannot be". The run wrote the explanation into the table cell.

Proposed fix — not applied: add the third branch — a dimension whose derived Invariant is empty owes no AC; the table says "Invariant empty" and that discharges it.

G4 — The base title template reads wrongly in lab mode

Base §6 pins # Spec: <Project> — **Standalone Takeaway**. The overlay does not override it, so base wins under precedence rule 2 — but in this mode nobody is taking anything away; the lab fixes the assignment and the learner builds it there.

Proposed fix — not applied: either an OVERRIDE 3 on the title line, or an explicit note that the base title stands and why. Right now it is left to each run to notice and live with.

G5 — The overlay does not carry its own commit, but ADD 3 requires pinning it

ADD 3: "The provenance header pins both the base guide's commit and this overlay's." The overlay states only "Written against base guide 8e44ecb" — it has no self-hash. The run derived it from git log and got the right answer, but that is an inference about repo state, not something the document supplies.

Proposed fix — not applied: say how to obtain it — pin the overlay's commit as reported by git log -1 --format=%h -- <this file> — rather than requiring a fact the file does not contain.

G6 — Unclear whether environments/coding-agent-lab.md may be opened

The rules file says "Both are distilled from environments/coding-agent-lab.md", and OUT OF SCOPE forbids reading certain of its sections as instructions — which implies the file may be opened. The second run did not open it, judging the two overlay files self-contained. If the distillation dropped anything load-bearing, that run silently missed it.

Proposed fix — not applied: state it outright. Recommended: the source doc is provenance, not an input — do not read it during generation; if the overlay is missing a fact, that is an overlay bug to file, not a gap to fill by reading around it.

---

Related: #28 (base-guide findings from the same runs), #29 (the slides glob). Overlay defects already fixed: 58ecfb9.

🤖 Generated with Claude Code

Đọc .claude/skills/generate-spec/references/spec-generation-guide.coding-agent-lab.md và tệp .runtime.md đi kèm, sau đó xem xét sáu thay đổi overlay được đề xuất cùng các quy tắc ưu tiên hiện có. Chạy /generate-spec --env=coding-agent-lab trong một thiết lập clean-room để so sánh spec được tạo. Được xem là hoàn tất khi cả hai tệp overlay đều chứa sáu bản sửa lỗi đã thống nhất và một lần tạo lại chứng minh rằng từng bản đều được áp dụng.