microsoft / microsoft/hve-core
feat: Run skill-bench against existing hve-core skills to establish effectiveness baselines
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 301
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 92
Description
Part of the skill-bench integration epic: #1504
Depends on #1507 (npm integration complete).
Execute the bench against the current hve-core skill library to produce baseline effectiveness metrics and surface gaps where new or improved skills are needed.
**Scope:**
- Run bench in EVAL and COMPARE modes against all skills in production collections
- Identify skills scoring 0% baseline (requiring skill investment) vs. patterns learnable without skills
- Surface any undocumented architectural decisions revealed by failing expectations (candidates for ADRs or instructions)
**Acceptance criteria:**
- Baseline pass-rate table published for each skill
- Skills with 0% baseline (without-skill) flagged as candidates for new skills or skill improvements
- At least one ADR or instruction issue created for any design decision exposed by failing expectations in both conditions
Contributor guide
Research direction
Start with the skill-bench integration epic in #1504 and confirm the npm integration dependency in #1507. Run EVAL and COMPARE against all skills in the production collections, then publish a per-skill baseline pass-rate table and create follow-up skill or ADR/instruction issues for the identified gaps and design decisions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100