microsoft / microsoft/GitHub-Copilot-for-Azure
Remove deprecated Waza, Jest, and TriggerMatcher infrastructure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
Background & Motivation
Once all migrations are validated and the unified CI pipeline is running, the old Waza infrastructure, Jest integration tests, and TriggerMatcher utility must be fully removed. Leaving deprecated code in the repo increases confusion, maintenance burden, and onboarding friction for new contributors.
User Story
As a Platform Engineer, I want all deprecated evaluation infrastructure removed so that new contributors see one evaluation framework and one CI pipeline, eliminating confusion about which system to use.
Solution Approach
- Pre-flight validation: Before deleting anything, run the full Vally eval suite and confirm all migrated evals pass. Run the migration validation script from Issue 4 to confirm zero discrepancies.
- Waza removal (FR-E1, FR-E4):
- Delete all
tasks/*.yamldirectories under old eval locations - Delete
.waza.yamlproject config - Remove Waza-related npm scripts from
package.json - Remove any Waza bridge scripts
- Delete all
- Jest integration test removal (FR-E2):
- Delete all
integration.test.tsfiles fromtests/<skill>/ - Update Jest config to exclude removed test paths (FR-E6)
- Keep
tests/<skill>/unit.test.tsandtests/utils/(minus deprecated utilities)
- Delete all
- TriggerMatcher retirement (FR-E3):
- Delete all
triggers.test.tsfiles fromtests/<skill>/ - Delete
tests/utils/TriggerMatcher.ts - Delete associated Jest snapshots (
__snapshots__/)
- Delete all
- CI cleanup (FR-E5):
- Remove old
eval.yml/ Waza CI workflow - Remove old Jest integration CI workflow (or the integration-specific jobs)
- Ensure the new
evals.ymlfrom Issue 4 is the sole eval CI pipeline
- Remove old
- Documentation (FR-E7, FR-E8):
- Update
tests/README.mdto reflect that LLM evals live inevals/, unit tests intests/ - Archive
EVAL-MIGRATION-REPORT.md(move todocs/archive/)
- Update
- Validation: After deletion, run
vally eval --suite fulland all remaining Jest unit tests to confirm nothing is broken.
Files to delete:
.waza.yaml
evals/*/tasks/ (old Waza task directories)
tests/*/eval/ (old Waza eval directories under tests/)
tests/*/integration.test.ts (×38 files)
tests/*/triggers.test.ts (×34 files)
tests/*/evals/ (old nested eval directories)
tests/*/__snapshots__/triggers.* (trigger test snapshots)
tests/utils/TriggerMatcher.ts
.github/workflows/eval.yml (old Waza CI workflow, if separate)
Files to update:
package.json (remove Waza npm scripts)
jest.config.ts (exclude removed test paths)
tests/README.md (update to reflect new structure)
.github/workflows/ (remove integration test jobs from old workflows)
Acceptance Criteria
- FR-E1: All Waza task files deleted after migration validation confirms equivalence
- FR-E2: All Jest integration test files deleted after migration validation
- FR-E3: TriggerMatcher utility and all Jest trigger test files deleted
- FR-E4:
.waza.yaml, bridge scripts, and Waza npm scripts removed frompackage.json - FR-E5: Old CI workflows replaced by the unified
evals.ymlpipeline - FR-E6: Jest config updated to exclude removed test paths — remaining unit tests still pass
- FR-E7:
tests/README.mdupdated to document that LLM evals live inevals/and unit tests stay intests/ - FR-E8:
EVAL-MIGRATION-REPORT.mdarchived todocs/archive/ - Zero grep hits for "waza" in the repo (excluding docs/archive) — per PRD success metric
- Zero grep hits for "TriggerMatcher" in the repo (excluding docs/archive) — per PRD success metric
-
vally eval --suite fullpasses after cleanup - All remaining Jest unit tests pass after cleanup
- No orphaned fixture directories or config files remain
Out of Scope
- Writing new test cases to expand coverage (PRD §8)
- Migrating Jest unit tests to any other framework (PRD §8)
- Changes to SKILL.md files (PRD §8)
Testing Expectations
- Run
vally eval --suite full— all evals pass. - Run remaining Jest unit tests — all pass.
- Run
grep -r "waza" . --include="*.ts" --include="*.yaml" --include="*.yml" --include="*.json"— zero hits outsidedocs/archive/. - Run
grep -r "TriggerMatcher" . --include="*.ts"— zero hits. - Verify CI pipeline triggers correctly on a test PR after old workflows are removed.
Constraints
- All deletions must happen AFTER migration validation passes (PRD success metric: 0 cases lost).
- Jest config changes must not break remaining unit tests.
Related Issues
- Depends on Issue 1 — Waza eval configs must be migrated before Waza files can be deleted.
- Depends on Issue 2 — Jest integration tests must be migrated before deletion.
- Depends on Issue 3 — Trigger tests and TriggerMatcher must be migrated before retirement.
- Depends on Issue 4 — Unified CI pipeline must be validated before old CI workflows are removed.
- This is the final phase — no other issues depend on it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.