VeryGoodOpenSource / VeryGoodOpenSource/vgv-wingspan

feat: add evaluation tables and data to test skills against updates

Open
#200 0 comments 0 reactions 1 assignee View on GitHub

@ryzizub is already working on this.

Since Jul 27, 2026.

feature p1
Dominant language
Shell
Stars
105
Forks
15
Avg merge
13h 12m
Merged PRs (30d)
2

Description

Description

Add evaluation tables and reference data so Wingspan skills can be tested against Claude Code updates. Today there is no structured way to verify a skill still behaves correctly after a Claude Code change. Define eval cases (inputs and expected outcomes) per skill so regressions are caught automatically rather than discovered in use.

Requirements

  • Define an eval data format (input scenario, expected behavior/output, pass criteria) reusable across skills.
  • Store eval tables/data co-located with each skill or under a shared evals/ directory.
  • Seed eval cases for the core workflow skills (/brainstorm, /plan, /build, /review).
  • Provide a runner that executes the evals and reports pass/fail.
  • Document how to add a new eval case when adding or changing a skill.

Additional Context

Consumed by the weekly changelog-check bot: when a skill is flagged as potentially impacted, its eval table is the gate for confirming it still works. See #199.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.