https-deeplearning-ai / https-deeplearning-ai/spec-build-lab
Next pass: candidate slate (A) + measurement watchlist (B)
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- Python
- Sterne
- 3
- Forks
- 3
- Ø Merge
- 20 Std. 23 Min.
- Gemergte PRs (30 T.)
- 6
Beschreibung
Forward-looking slate after PR #25. Two lists with different triggers: A needs a promotion pass (guide/spec edits + a validation round); B needs a tester round or a trial, not edits. Nothing here is applied.
Supersedes the forward halves of #18 and #24 (both closed). #19 (recording/compaction) stays separate — it is tooling, not spec/guide.
A. Candidates for the next promotion pass
Ordered by strength of evidence.
-
Harness-known identities are injected, never model-guessed. The agent-memory spec has this as an R11 sentence; the guide does not. Evidence it is needed: a clean-room regen against the current guide (regen-D) contains zero such language, so a spec generated today ships the same gap that made AC21 fail — the model, never told its thread id, guessed
"current_thread"and the summarize tool honestly consolidated nothing. Second datapoint: the first wording named one tool, and an independent trial build applied it to exactly that tool, leaving summary expansion unscoped. Write it concept-anchored (identities the harness knows are injected and absent from the model-facing schema; identifiers the model legitimately holds stay model-supplied). -
A shipped default that no acceptance criterion exercises (D10 augmentation). Tool-description augmentation is ON by default, but the offline stub cannot return augmentation JSON and no
liveAC registers through a real model — so 2 of 2 builds shipped it unexercised. Compare R5, which the spec deliberately made stub-testable. Candidate rule: a Ledger default that changes behavior needs at least one AC that exercises that default, offline-testable or explicitlylive. -
An AC depending on a fixture the spec never declares. AC7 and AC11 require a "scripted summarizer stub"; §5 declares four stubs and that is not one of them — in a section that opens "MUST create these fixtures exactly as described." 2 of 2 builds invented it, so part of the offline oracle rests on builder-authored fixture content. Candidate: §14 audit line — every fixture an AC names is declared in §5.
-
Contract constraints with no AC behind them.
SummaryRecord.descriptionminLength 8 and R6's 8–12-word label band are stated in prose/schema and never asserted. Candidate: the existing rules↔ACs bidirectional audit extends to schema constraints that encode behavior. -
Prose-economy sweep. This batch grew the guide 11,041 → 12,313 words (+11.5%) and the spec 9,773 → 10,528 (+7.7%), concentrated in §0/§6.0 — the sections three testers called too dense. At this rate the guide doubles roughly every six batches. Candidate: a pass whose goal is subtraction (merge overlapping rules, move rationale out of emitted templates, delete restated mechanics) rather than new rules, with the regen properties as the safety net.
-
F8 — fixture-corpus churn across regenerations. Parked by owner decision, carried here so it is not lost. Each regeneration authors fresh synthetic fixtures by design, so fixture identities churn and cross-generation comparisons carry noise. Not currently biting: with the Option-B pattern (surgical spec edits, no regen adoption) regenerations are evidence, not products.
B. Measurement watchlist
Needs a tester round or a disposable-clone trial. No edits pending.
- Does the orientation block cure the density complaint? Three testers found the spec overwhelming up front (R16: "a brief, high-level overview at the top"; R18: "I needed a better explanations and instructions in the beginning"). The fix shipped in #25 and is unproven with humans — a build agent reusing the prose is not evidence.
- Does the one-question gate feel light enough? R17 objected to "the heavy question-tool ceremony", and F12 replaced 13 questions with one for the default path. Needs a human reaction, not a probe.
- Baseline resource benchmark (wall-clock, tokens, disk) for a recommended-baseline build. This is the answer to the cost-abandonment feedback, which the owner decided to measure rather than promise in spec text.
- Does a build actually write
resolved-decisions.md? Accepted residual from #25: the requirement is validated in text, the behavior is not. Rides the next trial rather than a paid probe build. - Ingestion idempotency — one tester hit UNIQUE errors re-running setup. The current build cannot crash that way (generated ids) but duplicates silently. Proposal recorded, unapplied: an idempotent-seed AC or an explicit "re-seeding duplicates; reset first" note.
- arXiv 429 recurrence — transient rate limits hit a tester and our own runs; the client retries internally and R11 makes a mid-loop 429 non-fatal. Proposal recorded, unapplied: document transient-429 as expected for
live (net)criteria rather than mandate backoff code.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Behandle dies als eine Planungs- und Messbeobachtungsliste statt als einzelne Einstiegsaufgabe. Lies PR #25 sowie die agent-memory-Spezifikation und den agent-memory-Leitfaden und vergleiche anschließend regen-D, AC7/AC11, D10 sowie die Audits nach §5/§14. Ein Follow-up ist bereit, wenn ein Kandidat über einen klar begrenzten Änderungs- und Validierungsplan verfügt; Elemente der Beobachtungsliste erfordern eine Testerrunde oder einen Versuch mit einem kurzlebigen Klon.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Bereich
- documentation, testing
- Issue-Typ
- Dokumentation
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Aktiv
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100