Azure / Azure/git-ape

CI: static lint pipeline + strict JSON Schemas for state.json and sibling artifacts

Open
#55 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
269
Forks
45
Avg merge
18h 41m
Merged PRs (30d)
11

Description

## Background

Today the PR pipeline only checks docs staleness, workflow lint (`actionlint`), and plugin version drift. It does **not** validate that:

- Bash scripts (`.github/scripts/`, `.github/skills/**/scripts/`) are `shellcheck`-clean
- YAML files conform to a consistent style
- Markdown is consistent
- The seven JSON artifact types Git-Ape emits per deployment match a documented shape:
- `state.json`, `metadata.json`, `requirements.json`, `cost-estimate.json`, `security-gate.json`, `policy-recommendations.json`, `parameters.json`

Real-world drift exists. Inspecting an existing artifact corpus (e.g. `Azure/git-ape-private`) shows two distinct, incompatible shapes for `security-gate.json` shipping under the same claimed `schemaVersion: "1.0"`:

- **Boolean form** (older): `criticalPassed: true`, `highPassed: true`
- **Count form** (newer): `criticalTotal: 5`, `criticalPassed: 5`, `highTotal: 5`, `highPassed: 5`

Same field name, different type. Only `state.json` carries `schemaVersion` at all today.

## Goal

Land a tiered static-validation pipeline so every PR catches the broadest set of regressions before merge — without requiring a sandbox subscription. This issue covers Phase 1 (static lint), Phase 1A (schema introduction), and a reduced Phase 2 (fixture-driven schema validation).

### Phase 1 — Static lint pipeline

New `.github/workflows/git-ape-ci.yml` running in parallel jobs:

- `shellcheck` on every `.sh` (strict mode)
- `yamllint` on every YAML
- `markdownlint` on every `.md` (excluding generated `website/build`)
- `check-jsonschema` validation of every committed JSON artifact (see Phase 1A)
- `bats` tests against the fixture corpus

Reuse (do not duplicate):

- `git-ape-actionlint.yml` — already covers workflow lint
- `git-ape-docs-check.yml` — already covers docs staleness
- `git-ape-plugin-version-check.yml` — already covers `plugin.json` version drift

### Phase 1A — Schema introduction & versioning

Author strict JSON Schemas (draft 2020-12) under `schemas/git-ape//v1.json`:

- `_defs/v1.json` — shared types (`deploymentId`, `azureSubscription`, `armResourceId`, `iso8601DateTime`, `cafAbbreviation`, `environmentTier`, `deploymentStatus`, `softDeletableType`)
- `state/v1.json` — matches `website/docs/deployment/state.md`
- `metadata/v1.json`, `security-gate/v1.json`, `requirements/v1.json`, `cost-estimate/v1.json`, `policy-recommendations/v1.json`
- `plugin/v1.json` — for `plugin.json`

Locked-in design decisions:

1. **Versioning model**: per-artifact `schemaVersion`, shared `$defs` versioned together as a release train
2. **Strictness**: `additionalProperties: false` on top-level objects only; nested objects allow extras during transition
3. **Pre-1.0 corpus**: migrate-first, quarantine fallback (migration tooling shipped in this PR)
4. **`security-gate.json`**: adopts count form only; boolean form is rejected at v1.0
5. **`$schema` injection**: emitters write a relative path (no network coupling)
6. **Doc generation**: hybrid — narrative hand-written, field tables generated in a follow-up

### Phase 2 — Reduced (parity deferred)

`main` does not yet have a `state.json` emitter; those scripts arrive in #44. Phase 2 in this PR therefore covers:

- Fixture corpus under `tests/fixtures/` with valid + invalid samples per artifact type
- `bats` tests asserting good fixtures pass schema validation and bad fixtures fail
- Negative test for `security-gate.json` boolean form (must be rejected by v1.0 schema)

Bash↔PowerShell parity testing is deferred to a follow-up issue once #44 lands.

## Out of scope (tracked separately)

- Real-Azure sandbox E2E (`/e2e` label trigger) — Phase 4
- Agent behavioral snapshot tests — Phase 5
- PR feedback aggregation — Phase 6
- bash↔PowerShell parity (depends on #44)
- ARM template static checks (Checkov, ARM-TTK, PSRule for Azure, MSDO templateanalyzer)
- Auto-generated field-reference docs

## Acceptance criteria

- [ ] `git-ape-ci.yml` runs all jobs in parallel and gates the PR
- [ ] At least 7 schema files committed under `schemas/git-ape/`
- [ ] Shared `$defs` referenced from per-artifact schemas (no duplication)
- [ ] `scripts/validate-schemas.sh` invokable locally and from CI
- [ ] Negative-test fixtures (e.g., `security-gate.json` boolean form, `state.json` missing required field) cause CI to fail when run against the strict schemas
- [ ] `plugin.json` itself validates against `schemas/git-ape/plugin/v1.json`
- [ ] All tools pinned to versions (no floating)
- [ ] `tests/fixtures/README.md` documents how to add a new fixture

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.