Next pass: candidate slate (A) + measurement watchlist (B)

Abierto
#26 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Documentación
Claridad
Necesita aclaración
Estado de actividad
Activo

Línea de trabajo

Trata esto como una lista de seguimiento de planificación y medición, no como una única tarea inicial. Lee PR #25 y la especificación y la guía de agent-memory; después compara regen-D, AC7/AC11, D10 y las auditorías de §5/§14. Un seguimiento está listo cuando un candidato tiene un plan acotado de edición y validación; los elementos de la lista de seguimiento requieren una ronda de pruebas o un ensayo con un clon desechable.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Forward-looking slate after PR #25. Two lists with different triggers: A needs a promotion pass (guide/spec edits + a validation round); B needs a tester round or a trial, not edits. Nothing here is applied.

Supersedes the forward halves of #18 and #24 (both closed). #19 (recording/compaction) stays separate — it is tooling, not spec/guide.

A. Candidates for the next promotion pass

Ordered by strength of evidence.

  1. Harness-known identities are injected, never model-guessed. The agent-memory spec has this as an R11 sentence; the guide does not. Evidence it is needed: a clean-room regen against the current guide (regen-D) contains zero such language, so a spec generated today ships the same gap that made AC21 fail — the model, never told its thread id, guessed "current_thread" and the summarize tool honestly consolidated nothing. Second datapoint: the first wording named one tool, and an independent trial build applied it to exactly that tool, leaving summary expansion unscoped. Write it concept-anchored (identities the harness knows are injected and absent from the model-facing schema; identifiers the model legitimately holds stay model-supplied).

  2. A shipped default that no acceptance criterion exercises (D10 augmentation). Tool-description augmentation is ON by default, but the offline stub cannot return augmentation JSON and no live AC registers through a real model — so 2 of 2 builds shipped it unexercised. Compare R5, which the spec deliberately made stub-testable. Candidate rule: a Ledger default that changes behavior needs at least one AC that exercises that default, offline-testable or explicitly live.

  3. An AC depending on a fixture the spec never declares. AC7 and AC11 require a "scripted summarizer stub"; §5 declares four stubs and that is not one of them — in a section that opens "MUST create these fixtures exactly as described." 2 of 2 builds invented it, so part of the offline oracle rests on builder-authored fixture content. Candidate: §14 audit line — every fixture an AC names is declared in §5.

  4. Contract constraints with no AC behind them. SummaryRecord.description minLength 8 and R6's 8–12-word label band are stated in prose/schema and never asserted. Candidate: the existing rules↔ACs bidirectional audit extends to schema constraints that encode behavior.

  5. Prose-economy sweep. This batch grew the guide 11,041 → 12,313 words (+11.5%) and the spec 9,773 → 10,528 (+7.7%), concentrated in §0/§6.0 — the sections three testers called too dense. At this rate the guide doubles roughly every six batches. Candidate: a pass whose goal is subtraction (merge overlapping rules, move rationale out of emitted templates, delete restated mechanics) rather than new rules, with the regen properties as the safety net.

  6. F8 — fixture-corpus churn across regenerations. Parked by owner decision, carried here so it is not lost. Each regeneration authors fresh synthetic fixtures by design, so fixture identities churn and cross-generation comparisons carry noise. Not currently biting: with the Option-B pattern (surgical spec edits, no regen adoption) regenerations are evidence, not products.

B. Measurement watchlist

Needs a tester round or a disposable-clone trial. No edits pending.

  • Does the orientation block cure the density complaint? Three testers found the spec overwhelming up front (R16: "a brief, high-level overview at the top"; R18: "I needed a better explanations and instructions in the beginning"). The fix shipped in #25 and is unproven with humans — a build agent reusing the prose is not evidence.
  • Does the one-question gate feel light enough? R17 objected to "the heavy question-tool ceremony", and F12 replaced 13 questions with one for the default path. Needs a human reaction, not a probe.
  • Baseline resource benchmark (wall-clock, tokens, disk) for a recommended-baseline build. This is the answer to the cost-abandonment feedback, which the owner decided to measure rather than promise in spec text.
  • Does a build actually write resolved-decisions.md? Accepted residual from #25: the requirement is validated in text, the behavior is not. Rides the next trial rather than a paid probe build.
  • Ingestion idempotency — one tester hit UNIQUE errors re-running setup. The current build cannot crash that way (generated ids) but duplicates silently. Proposal recorded, unapplied: an idempotent-seed AC or an explicit "re-seeding duplicates; reset first" note.
  • arXiv 429 recurrence — transient rate limits hit a tester and our own runs; the client retries internally and R11 makes a mid-loop 429 non-fatal. Proposal recorded, unapplied: document transient-429 as expected for live (net) criteria rather than mandate backoff code.
Lenguaje dominante
Python
Estrellas
3
Forks
3
Merge medio
20 h 23 min
PR fusionados (30 d)
6

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de https-deeplearning-ai/spec-build-lab

Todos los issues de https-deeplearning-ai/spec-build-lab

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.