addyosmani / addyosmani/agent-skills

documentation-and-adrs: no mechanism for detecting doc/code drift

未關閉
#511 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
JavaScript
星號
93.8k
分支
10k
平均合併
3 天 15 小時
30 天內合併 PR
23

描述

`documentation-and-adrs` covers writing ADRs and changelogs well, but nothing in the pack detects when documentation and the code it describes have diverged. Documentation rot is a distinct failure from missing documentation, and agents accelerate it — they generate docs readily and have no mechanism to notice when those docs stop being true.

### The gap

I scanned the current `SKILL.md` (1,470 words) for the concepts this would cover: `drift`, `stale`, `last verified`, `append-only`, `change log`, `pre-push`, `doc rot`, `out of sync`. Zero hits on all eight. The two near-misses are different things — `## Changelog Maintenance` is release notes for users, and the "outdated" discussion is about code comments going stale, not documentation diverging from the code it describes.

Duplicate check per CONTRIBUTING: I went through open PRs and issues and found no claim on this. The closest three are adjacent but distinct — #103 covers the intent-to-verification traceability chain, #398 preserves durable knowledge as memory capsules, and #432's "drift" is model drift across a release, a different sense of the word.

### Proposal

As a section in `documentation-and-adrs` rather than a new directory, per rule 4's preference for extending an existing skill:

- a `Last verified:` stamp on each doc
- an append-only change log, so history is never overwritten by a rewrite
- a routing file mapping each code path to the doc that must change with it, so a check can flag any module whose doc is missing, or whose last commit is older than its code's

The routing file is what makes this mechanical rather than advisory: without it, "keep docs current" has nothing to check against.

### Implementation notes, honestly scoped

I run this as a Python CLI (ADDA), currently on exactly one repository — its own — and not yet public, so treat this as one implementation rather than a proven pattern at scale.

On its first run it reported an eight-module documentation hole that had gone unnoticed for two months. The previous regime there was a written instruction to keep module docs current, with nothing enforcing it, and the directory simply stayed empty.

Two design choices worth stating because both have real trade-offs:

**The commit hook blocks (exit 1) rather than warning**, with `--no-verify` and an env-var escape. Blocking creates friction and will get bypassed under deadline. But warn-only gets scrolled past, and a warn-only-equivalent regime is precisely what produced the two-month hole above. I don't think there's a free option here.

**Staleness is decided by git commit ancestry, not timestamps.** Commit times have one-second resolution, so a doc and its code committed in the same second tie and the doc reads as current. Ancestry also has to be three-state: when neither commit is an ancestor of the other — divergent branches, rebase, cherry-pick — the honest answer is "cannot tell", and reporting that as "current" is a silent pass.

Happy to write this to `skill-anatomy.md` format with eval cases if it's wanted.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。