microsoft / microsoft/hve-core
feat: Integrate skill-bench with Prompt Builder workflow and npm scripts
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 301
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 92
Description
Part of the skill-bench integration epic: #1504
Depends on #1508 (core collection created).
Wire the skill-bench into the existing hve-core toolchain so it is runnable from the command line and optionally in CI.
**Scope:**
- Add `bench:run` and `bench:report` npm scripts to `package.json`
- Integrate bench EVAL mode into the Prompt Builder → Prompt Evaluator → Prompt Updater cycle so eval results can feed the authoring feedback loop
- Document CI output format decision (JUnit XML, TAP, or custom JSON matching existing `logs/` patterns)
**Acceptance criteria:**
- `npm run bench:run` executes the bench orchestrator
- `npm run bench:report` outputs a summary of results
- Integration with Prompt Evaluator is documented (even if fully automated wiring is deferred)
- Scripts work on both Windows (PowerShell) and Unix (bash)
Contributor guide
Research direction
Start with package.json and locate the existing bench orchestrator, Prompt Builder, Prompt Evaluator, and Prompt Updater entry points. Check dependency #1508 before wiring the workflow. Done means both bench scripts run on PowerShell and bash, report results, and document the evaluator integration and chosen CI output format.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, cli, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100