microsoft / microsoft/hve-core
docs: update ci:eval:lint:vally command references for grader-lineage check
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 301
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 92
Description
Component
Documentation
Bug Description
commit 56c30bcb (#2934) changed the ci:eval:lint:vally npm script in package.json from:
pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/
to:
pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && npm run lint:eval-grader-lineage && vally lint --eval-spec evals/
adding a new lint:eval-grader-lineage step (Build-GraderLineageMap.ps1 -Check) in between. Two documentation pages still show the pre-change command composition and do not mention the new grader-lineage check or script:
docs/architecture/workflows.md(script/workflow table row forci:eval:lint:vally)docs/contributing/evals-ci.md(script/workflow table and the prose describingci:eval:lint:vally runs vally lint --eval-spec evals/ ...)
Expected Behavior
Both pages should describe the current three-step composition of ci:eval:lint:vally, including the lint:eval-grader-lineage step and a brief mention of what scripts/evals/Build-GraderLineageMap.ps1 -Check validates (grader lineage mapping), so the documented command matches package.json.
Steps to Reproduce
- See
package.jsonchange in commit 56c30bcb / PR #2934. - Compare against
docs/architecture/workflows.mdline ~445 anddocs/contributing/evals-ci.mdline ~280, which still describe only theBuild-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/composition.
Additional Context
Related files:
package.json(ci:eval:lint:vally,lint:eval-grader-lineagescripts)scripts/evals/Build-GraderLineageMap.ps1docs/architecture/workflows.mddocs/contributing/evals-ci.md
Generated by Documentation Update Check · auto · 84.6 AIC · ⌖ 1.73 AIC · ⊞ 9.5K · ◷
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare package.json with the command descriptions in docs/architecture/workflows.md and docs/contributing/evals-ci.md. Read the lint:eval-grader-lineage script and scripts/evals/Build-GraderLineageMap.ps1 -Check to confirm what the new validation covers. Done when both pages describe all three command steps and the grader-lineage validation accurately.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- powershell
- Domain
- ci-cd, documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100