Add generate-sdk-locally trigger eval and verify skill pipeline
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 130
Description
## Context
PR #15376 completed the Waza-to-Vally migration tracked by #15147. One narrow cleanup item remains: `azsdk-common-generate-sdk-locally` has its capability eval but is the only originally migrated skill without a routing eval.
Broader eval-coverage enforcement remains tracked by #16345.
## Tasks
- Add `.github/skills/azsdk-common-generate-sdk-locally/evals/trigger.eval.yaml`.
- Include representative trigger and anti-trigger prompts.
- Follow the existing `skill-invocation.required` and `skill-invocation.disallowed` conventions.
- Confirm the eval configuration passes Vally lint.
- Link one Azure DevOps skill-eval run where the affected skill shard completes with no failed evals.
## Acceptance criteria
- [ ] Trigger and anti-trigger routing coverage exists for `azsdk-common-generate-sdk-locally`.
- [ ] Vally lint succeeds.
- [ ] A linked Azure DevOps run shows the affected skill evals passing. A report-only or `continueOnError` aggregate success with failed tests underneath is not sufficient.
## Out of scope
- General skill/MCP coverage enforcement tracked by #16345.
- Plugin-tree Waza assets.
- Unrelated eval-quality work.
Contributor guide
Research direction
Start by reading the existing capability eval for .github/skills/azsdk-common-generate-sdk-locally and the skill-invocation.required and skill-invocation.disallowed conventions. Add trigger and anti-trigger coverage in evals/trigger.eval.yaml, run Vally lint, and link an Azure DevOps skill-eval run showing the affected shard passing without failed evals.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- yaml
- Domain
- testing, tooling
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100