microsoft / microsoft/hve-core
docs: scripts/README.md is missing entries for 13 evals scripts added in feat(mcsb)
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 301
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 92
Description
## Component
Documentation
## Bug Description
The `scripts/README.md` Evals section documents only 7 of the 20 scripts in `scripts/evals/`. The following 13 scripts were added or are present in the repository but have no entry in the reference table:
| Script | Synopsis |
|--------|----------|
| `Get-AgentDependencyMap.ps1` | Build a JSON map of agent dependencies for the baseline-equivalence dispatcher |
| `Get-ChangedAIArtifact.ps1` | Emit a JSON manifest of AI customization artifacts changed between two git refs |
| `Get-ChangedSpecStimulus.ps1` | Emit a JSON manifest of synthetic artifacts derived from changed eval specs |
| `Invoke-ArtifactModeration.ps1` | Content-moderation pre-job for all eval specs plus changed AI artifacts |
| `New-AgentMatrixDashboard.ps1` | Render a self-contained HTML matrix dashboard for the agent-behavior eval suite |
| `New-AgentSurfaceSignatures.ps1` | Generate per-agent surface signature YAML for baseline equivalence runs |
| `New-EquivalenceDashboard.ps1` | Render a self-contained HTML dashboard for a local baseline-equivalence eval run |
| `Test-CopilotToken.ps1` | Pre-flight probe for the `COPILOT_GITHUB_TOKEN` secret used by vally evals |
| `Test-EvalSpec.ps1` | Validate vally eval spec files against the embedded schema and enforce per-agent behavioral eval coverage |
| `Test-EvalSpecText.ps1` | Run alex.js and retext-profanities against the AI-artifact markdown corpus |
| `Test-StimulusPresence.ps1` | Verify every changed AI artifact has at least one matching eval-spec stimulus backlink |
| `Test-VallyTestSafety.ps1` | Repo-wide safety lint that flags eval stimuli and corpora matching the skill-local refusal taxonomy |
| `Update-AgentMatrixSummariesFromLogs.ps1` | Rebuild per-agent matrix JSON summaries from existing vally logs without re-running `npx vally` |
## Expected Behavior
All scripts present in `scripts/evals/` should appear in the `## Evals` table in `scripts/README.md` with a concise purpose description, matching the pattern used for other script categories.
## Steps to Reproduce
The gap was introduced by the commit that added `feat(mcsb): add Microsoft Cloud Security Benchmark security skill (#2499)`, which included a large set of evals infrastructure scripts but did not add corresponding rows to `scripts/README.md`.
1. Run: `ls scripts/evals/*.ps1`
2. Compare against the `## Evals` table in `scripts/README.md`
3. Observe that only 7 of the 20 scripts appear in the table
## Additional Context
- Documentation file: [`scripts/README.md`](scripts/README.md)
- Code directory: [`scripts/evals/`](scripts/evals/)
- All 13 missing scripts include `Get-Help`-compatible `.SYNOPSIS` blocks that can be used verbatim as table entries
> Generated by [Documentation Update Check](https://github.com/microsoft/hve-core/actions/runs/30657069179) · sonnet46 60.4 AIC · ⌖ 5.88 AIC · ⊞ 7.8K · [◷](https://github.com/search?q=repo%3Amicrosoft%2Fhve-core+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2Fhve-core%2Fdoc-update-check%22&type=issues)
Contributor guide
Research direction
Start with the Evals section in scripts/README.md and compare its table with the output of ls scripts/evals/*.ps1. Read the Get-Help-compatible .SYNOPSIS blocks in the 13 listed scripts for concise descriptions. Done means all 20 scripts in scripts/evals/ have entries in the Evals table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- powershell
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100