VeryGoodOpenSource / VeryGoodOpenSource/vgv-wingspan
feat: add evaluation tables and data to test skills against updates
Open
@ryzizub is already working on this.
Since Jul 27, 2026.
feature
p1
- Dominant language
- Shell
- Stars
- 105
- Forks
- 15
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 2
Description
Description
Add evaluation tables and reference data so Wingspan skills can be tested against Claude Code updates. Today there is no structured way to verify a skill still behaves correctly after a Claude Code change. Define eval cases (inputs and expected outcomes) per skill so regressions are caught automatically rather than discovered in use.
Requirements
- Define an eval data format (input scenario, expected behavior/output, pass criteria) reusable across skills.
- Store eval tables/data co-located with each skill or under a shared
evals/directory. - Seed eval cases for the core workflow skills (
/brainstorm,/plan,/build,/review). - Provide a runner that executes the evals and reports pass/fail.
- Document how to add a new eval case when adding or changing a skill.
Additional Context
Consumed by the weekly changelog-check bot: when a skill is flagged as potentially impacted, its eval table is the gate for confirming it still works. See #199.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.