microsoft / microsoft/hve-core

docs: update ci:eval:lint:vally command references for grader-lineage check

Open Beginner friendly
#2,944 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

agent-ready documentation needs-triage
Dominant language
Python
Stars
1.5k
Forks
301
Avg merge
3d 3h
Merged PRs (30d)
92

Description

Component

Documentation

Bug Description

commit 56c30bcb (#2934) changed the ci:eval:lint:vally npm script in package.json from:

pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/

to:

pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && npm run lint:eval-grader-lineage && vally lint --eval-spec evals/

adding a new lint:eval-grader-lineage step (Build-GraderLineageMap.ps1 -Check) in between. Two documentation pages still show the pre-change command composition and do not mention the new grader-lineage check or script:

  • docs/architecture/workflows.md (script/workflow table row for ci:eval:lint:vally)
  • docs/contributing/evals-ci.md (script/workflow table and the prose describing ci:eval:lint:vally runs vally lint --eval-spec evals/ ...)
Expected Behavior

Both pages should describe the current three-step composition of ci:eval:lint:vally, including the lint:eval-grader-lineage step and a brief mention of what scripts/evals/Build-GraderLineageMap.ps1 -Check validates (grader lineage mapping), so the documented command matches package.json.

Steps to Reproduce
  1. See package.json change in commit 56c30bcb / PR #2934.
  2. Compare against docs/architecture/workflows.md line ~445 and docs/contributing/evals-ci.md line ~280, which still describe only the Build-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/ composition.
Additional Context

Related files:

  • package.json (ci:eval:lint:vally, lint:eval-grader-lineage scripts)
  • scripts/evals/Build-GraderLineageMap.ps1
  • docs/architecture/workflows.md
  • docs/contributing/evals-ci.md

Generated by Documentation Update Check · auto · 84.6 AIC · ⌖ 1.73 AIC · ⊞ 9.5K ·

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare package.json with the command descriptions in docs/architecture/workflows.md and docs/contributing/evals-ci.md. Read the lint:eval-grader-lineage script and scripts/evals/Build-GraderLineageMap.ps1 -Check to confirm what the new validation covers. Done when both pages describe all three command steps and the grader-lineage validation accurately.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
ci-cd, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
90/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.