addyosmani / addyosmani/agent-skills
Drift data for 3 of these skills across the sonnet-4-6 → sonnet-5 release
- Ngôn ngữ chính
- JavaScript
- Star
- 93.8k
- Fork
- 10k
- Merge trung bình
- 3 ngày 15 giờ
- Pull request đã merge (30 ngày)
- 23
Mô tả
Hi Addy — I run a small OSS project (Driftproof) that re-tests agent skills across model releases: same suite, with/without the skill, judge sampled 5x so every score carries a confidence band, and a regression is only claimed when bands separate AND the move clears a 0.05 effect floor.
For our first report I wrote eval suites for three skills from this repo (code-review-and-quality, git-workflow-and-versioning, documentation-and-adrs), each case grounded in a claim the skill's own SKILL.md makes, and ran them across the sonnet-4-6 → sonnet-5 release. Results, with receipts:
- code-review-and-quality: WITHIN NOISE (held up)
- git-workflow-and-versioning: 1 case regressed (semver-clean-bump)
- documentation-and-adrs: MIXED (one sharp regression, one improvement)
Full tables + methodology: https://driftproofhq.com/reports/001/
The suites themselves: https://github.com/driftproofhq/driftproof/tree/main/suites
Offer, take or leave any of it: the suites are Apache-2.0 and yours to adopt upstream if useful; `npx driftproof run` re-runs them on any model with your own key; and there's a GitHub Action + badge if continuous verification on releases is interesting. If you think any case tests something your skill doesn't actually claim, I'd genuinely like to know — suite fairness is the part I care most about getting right.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.