Azure / Azure/awesome-azd

Add automated `aicollection` tag validation to npm test

Open
#885 0 comments 0 reactions 0 assignees View on GitHub
data schema enhancement maintenance
Dominant language
TypeScript
Stars
269
Forks
176
Avg merge
11h 1m
Merged PRs (30d)
37

Description

## Problem

The `aicollection` tag in `website/static/templates.json` is reserved for templates listed in the Microsoft AI collection at https://azure.github.io/ai-app-templates/. Today we enforce that rule only through manual PR review.

This came up in #780, which proposes adding 36 templates all carrying the `aicollection` tag. Checking each entry by hand against the AI collection site is slow and error-prone, and at scale it blocks otherwise-clean PRs from getting merged.

## Proposed approach

Add a check to the existing `npm test` suite in `website/` that:

1. Fetches the canonical list of templates from https://azure.github.io/ai-app-templates/ (ideally from a JSON manifest, not by scraping HTML)
2. For each entry in `templates.json` carrying the `aicollection` tag, asserts the source repo URL (or another stable identifier) appears in the canonical list
3. Fails with a clear message that names the offending templates if any `aicollection`-tagged entry is missing from the canonical list

## Open questions

- Does the AI collection site expose a machine-readable manifest, or should we check in a local allow-list and refresh it on a cadence?
- How should we handle network failures in CI, soft-fail with a warning or skip the check entirely?
- Should this also cover other reserved tags like `msft`, `popular`, and `featured`?

## Acceptance criteria

- [ ] `npm test` fails when a template carries `aicollection` without a match in the canonical source
- [ ] The failure message names the mistagged templates
- [ ] CI behavior is documented in the contributor guide
- [ ] Existing `templates.json` passes the new check, or any known violations are tracked in a follow-up

## Context

Raised in the review of #780.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with website/static/templates.json and the existing npm test command in website. Investigate whether https://azure.github.io/ai-app-templates/ exposes a machine-readable manifest, then determine how network failures and reserved tags should be handled. Done means npm test reports offending template names, existing entries pass or are tracked, and the CI behavior is documented in the contributor guide.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
devtools, documentation, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.