Add automated `aicollection` tag validation to npm test
- Dominant language
- TypeScript
- Stars
- 269
- Forks
- 176
- Avg merge
- 11h 1m
- Merged PRs (30d)
- 37
Description
## Problem
The `aicollection` tag in `website/static/templates.json` is reserved for templates listed in the Microsoft AI collection at https://azure.github.io/ai-app-templates/. Today we enforce that rule only through manual PR review.
This came up in #780, which proposes adding 36 templates all carrying the `aicollection` tag. Checking each entry by hand against the AI collection site is slow and error-prone, and at scale it blocks otherwise-clean PRs from getting merged.
## Proposed approach
Add a check to the existing `npm test` suite in `website/` that:
1. Fetches the canonical list of templates from https://azure.github.io/ai-app-templates/ (ideally from a JSON manifest, not by scraping HTML)
2. For each entry in `templates.json` carrying the `aicollection` tag, asserts the source repo URL (or another stable identifier) appears in the canonical list
3. Fails with a clear message that names the offending templates if any `aicollection`-tagged entry is missing from the canonical list
## Open questions
- Does the AI collection site expose a machine-readable manifest, or should we check in a local allow-list and refresh it on a cadence?
- How should we handle network failures in CI, soft-fail with a warning or skip the check entirely?
- Should this also cover other reserved tags like `msft`, `popular`, and `featured`?
## Acceptance criteria
- [ ] `npm test` fails when a template carries `aicollection` without a match in the canonical source
- [ ] The failure message names the mistagged templates
- [ ] CI behavior is documented in the contributor guide
- [ ] Existing `templates.json` passes the new check, or any known violations are tracked in a follow-up
## Context
Raised in the review of #780.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with website/static/templates.json and the existing npm test command in website. Investigate whether https://azure.github.io/ai-app-templates/ exposes a machine-readable manifest, then determine how network failures and reserved tags should be handled. Done means npm test reports offending template names, existing entries pass or are tracked, and the CI behavior is documented in the contributor guide.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- devtools, documentation, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100