addyosmani / addyosmani/agent-skills

Proposal: a proof-of-done skill (evidence ledger for completion claims)

Đang mở
#528 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
JavaScript
Star
93.8k
Fork
10k
Merge trung bình
3 ngày 15 giờ
Pull request đã merge (30 ngày)
23

Mô tả

Pre-flight done: searched the catalog and open PRs. Closest are doubt-driven-development (reviews decisions, not completion claims) and #523 (audits skill content, not agent output). Apologies if I missed something.

The gap: "Verify, Don't Assume" is a core behavior of this pack, but the completion claim itself has no artifact. When the agent says "tests pass, no regressions", the human either re-reads the transcript or trusts the prose. With auto mode now the default in most tools, trust concentrates exactly there. And agents do report work as done that they never did; I've caught mine doing it.

The design. The agent closes a task with a claims ledger instead of prose. Each claim carries a tag and a pointer into the session trace:

VERIFIED: a receipt exists (command + output in the trace)
INFERRED: deduced from indirect evidence, reasoning stated
ASSERTED: no evidence

The key rule: VERIFIED is never self-reported. The trace is written by the harness, not the model, so a small deterministic auditor resolves each pointer and recomputes the predicate itself (exit code, expected output). No matching receipt, automatic downgrade to ASSERTED. And a task cannot close with ASSERTED claims on its acceptance criteria: verify, or state "could not verify X because Y".

The ledger gets committed next to the spec and plan, with a digest of the trace. Git history is already a Merkle DAG, so this makes the whole thing tamper-evident with zero new infrastructure.

One thing stays honestly out of reach: whether the chosen checks are the right checks for the criteria. That part is not mechanizable, so the ledger keeps it explicit and small, and that's what the human actually reviews. A few predicates instead of a whole transcript, which is also cheaper on tokens.

Prior art: [superpowers' verification-before-completion](https://github.com/obra/superpowers/blob/main/skills/verification-before-completion/SKILL.md) states the principle well (evidence before claims). The differences here are the ledger as a persistent artifact, the INFERRED middle tier, and the auditor sitting outside the agent's self-report.

Would this fit the Verify phase? Happy to draft the skill as a PR if useful, or fold it into an existing skill if that fits better.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.