microsoft / microsoft/hve-core

docs: scripts/README.md is missing entries for 13 evals scripts added in feat(mcsb)

Open Beginner friendly
#2,571 4 comments 0 reactions 0 assignees View on GitHub
agent-ready] documentation good first issue needs-triage
Dominant language
Python
Stars
1.5k
Forks
301
Avg merge
3d 3h
Merged PRs (30d)
92

Description

## Component

Documentation

## Bug Description

The `scripts/README.md` Evals section documents only 7 of the 20 scripts in `scripts/evals/`. The following 13 scripts were added or are present in the repository but have no entry in the reference table:

| Script | Synopsis |
|--------|----------|
| `Get-AgentDependencyMap.ps1` | Build a JSON map of agent dependencies for the baseline-equivalence dispatcher |
| `Get-ChangedAIArtifact.ps1` | Emit a JSON manifest of AI customization artifacts changed between two git refs |
| `Get-ChangedSpecStimulus.ps1` | Emit a JSON manifest of synthetic artifacts derived from changed eval specs |
| `Invoke-ArtifactModeration.ps1` | Content-moderation pre-job for all eval specs plus changed AI artifacts |
| `New-AgentMatrixDashboard.ps1` | Render a self-contained HTML matrix dashboard for the agent-behavior eval suite |
| `New-AgentSurfaceSignatures.ps1` | Generate per-agent surface signature YAML for baseline equivalence runs |
| `New-EquivalenceDashboard.ps1` | Render a self-contained HTML dashboard for a local baseline-equivalence eval run |
| `Test-CopilotToken.ps1` | Pre-flight probe for the `COPILOT_GITHUB_TOKEN` secret used by vally evals |
| `Test-EvalSpec.ps1` | Validate vally eval spec files against the embedded schema and enforce per-agent behavioral eval coverage |
| `Test-EvalSpecText.ps1` | Run alex.js and retext-profanities against the AI-artifact markdown corpus |
| `Test-StimulusPresence.ps1` | Verify every changed AI artifact has at least one matching eval-spec stimulus backlink |
| `Test-VallyTestSafety.ps1` | Repo-wide safety lint that flags eval stimuli and corpora matching the skill-local refusal taxonomy |
| `Update-AgentMatrixSummariesFromLogs.ps1` | Rebuild per-agent matrix JSON summaries from existing vally logs without re-running `npx vally` |

## Expected Behavior

All scripts present in `scripts/evals/` should appear in the `## Evals` table in `scripts/README.md` with a concise purpose description, matching the pattern used for other script categories.

## Steps to Reproduce

The gap was introduced by the commit that added `feat(mcsb): add Microsoft Cloud Security Benchmark security skill (#2499)`, which included a large set of evals infrastructure scripts but did not add corresponding rows to `scripts/README.md`.

1. Run: `ls scripts/evals/*.ps1`
2. Compare against the `## Evals` table in `scripts/README.md`
3. Observe that only 7 of the 20 scripts appear in the table

## Additional Context

- Documentation file: [`scripts/README.md`](scripts/README.md)
- Code directory: [`scripts/evals/`](scripts/evals/)
- All 13 missing scripts include `Get-Help`-compatible `.SYNOPSIS` blocks that can be used verbatim as table entries

> Generated by [Documentation Update Check](https://github.com/microsoft/hve-core/actions/runs/30657069179) · sonnet46 60.4 AIC · ⌖ 5.88 AIC · ⊞ 7.8K · [◷](https://github.com/search?q=repo%3Amicrosoft%2Fhve-core+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2Fhve-core%2Fdoc-update-check%22&type=issues)

Contributor guide

Open the contributing guide

Research direction

Start with the Evals section in scripts/README.md and compare its table with the output of ls scripts/evals/*.ps1. Read the Get-Help-compatible .SYNOPSIS blocks in the 13 listed scripts for concise descriptions. Done means all 20 scripts in scripts/evals/ have entries in the Evals table.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
90/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.