CI: static lint pipeline + strict JSON Schemas for state.json and sibling artifacts
- Dominant language
- JavaScript
- Stars
- 269
- Forks
- 45
- Avg merge
- 18h 41m
- Merged PRs (30d)
- 11
Description
## Background
Today the PR pipeline only checks docs staleness, workflow lint (`actionlint`), and plugin version drift. It does **not** validate that:
- Bash scripts (`.github/scripts/`, `.github/skills/**/scripts/`) are `shellcheck`-clean
- YAML files conform to a consistent style
- Markdown is consistent
- The seven JSON artifact types Git-Ape emits per deployment match a documented shape:
- `state.json`, `metadata.json`, `requirements.json`, `cost-estimate.json`, `security-gate.json`, `policy-recommendations.json`, `parameters.json`
Real-world drift exists. Inspecting an existing artifact corpus (e.g. `Azure/git-ape-private`) shows two distinct, incompatible shapes for `security-gate.json` shipping under the same claimed `schemaVersion: "1.0"`:
- **Boolean form** (older): `criticalPassed: true`, `highPassed: true`
- **Count form** (newer): `criticalTotal: 5`, `criticalPassed: 5`, `highTotal: 5`, `highPassed: 5`
Same field name, different type. Only `state.json` carries `schemaVersion` at all today.
## Goal
Land a tiered static-validation pipeline so every PR catches the broadest set of regressions before merge — without requiring a sandbox subscription. This issue covers Phase 1 (static lint), Phase 1A (schema introduction), and a reduced Phase 2 (fixture-driven schema validation).
### Phase 1 — Static lint pipeline
New `.github/workflows/git-ape-ci.yml` running in parallel jobs:
- `shellcheck` on every `.sh` (strict mode)
- `yamllint` on every YAML
- `markdownlint` on every `.md` (excluding generated `website/build`)
- `check-jsonschema` validation of every committed JSON artifact (see Phase 1A)
- `bats` tests against the fixture corpus
Reuse (do not duplicate):
- `git-ape-actionlint.yml` — already covers workflow lint
- `git-ape-docs-check.yml` — already covers docs staleness
- `git-ape-plugin-version-check.yml` — already covers `plugin.json` version drift
### Phase 1A — Schema introduction & versioning
Author strict JSON Schemas (draft 2020-12) under `schemas/git-ape//v1.json`:
- `_defs/v1.json` — shared types (`deploymentId`, `azureSubscription`, `armResourceId`, `iso8601DateTime`, `cafAbbreviation`, `environmentTier`, `deploymentStatus`, `softDeletableType`)
- `state/v1.json` — matches `website/docs/deployment/state.md`
- `metadata/v1.json`, `security-gate/v1.json`, `requirements/v1.json`, `cost-estimate/v1.json`, `policy-recommendations/v1.json`
- `plugin/v1.json` — for `plugin.json`
Locked-in design decisions:
1. **Versioning model**: per-artifact `schemaVersion`, shared `$defs` versioned together as a release train
2. **Strictness**: `additionalProperties: false` on top-level objects only; nested objects allow extras during transition
3. **Pre-1.0 corpus**: migrate-first, quarantine fallback (migration tooling shipped in this PR)
4. **`security-gate.json`**: adopts count form only; boolean form is rejected at v1.0
5. **`$schema` injection**: emitters write a relative path (no network coupling)
6. **Doc generation**: hybrid — narrative hand-written, field tables generated in a follow-up
### Phase 2 — Reduced (parity deferred)
`main` does not yet have a `state.json` emitter; those scripts arrive in #44. Phase 2 in this PR therefore covers:
- Fixture corpus under `tests/fixtures/` with valid + invalid samples per artifact type
- `bats` tests asserting good fixtures pass schema validation and bad fixtures fail
- Negative test for `security-gate.json` boolean form (must be rejected by v1.0 schema)
Bash↔PowerShell parity testing is deferred to a follow-up issue once #44 lands.
## Out of scope (tracked separately)
- Real-Azure sandbox E2E (`/e2e` label trigger) — Phase 4
- Agent behavioral snapshot tests — Phase 5
- PR feedback aggregation — Phase 6
- bash↔PowerShell parity (depends on #44)
- ARM template static checks (Checkov, ARM-TTK, PSRule for Azure, MSDO templateanalyzer)
- Auto-generated field-reference docs
## Acceptance criteria
- [ ] `git-ape-ci.yml` runs all jobs in parallel and gates the PR
- [ ] At least 7 schema files committed under `schemas/git-ape/`
- [ ] Shared `$defs` referenced from per-artifact schemas (no duplication)
- [ ] `scripts/validate-schemas.sh` invokable locally and from CI
- [ ] Negative-test fixtures (e.g., `security-gate.json` boolean form, `state.json` missing required field) cause CI to fail when run against the strict schemas
- [ ] `plugin.json` itself validates against `schemas/git-ape/plugin/v1.json`
- [ ] All tools pinned to versions (no floating)
- [ ] `tests/fixtures/README.md` documents how to add a new fixture
Contributor guide
Assessment
This issue has not been assessed yet.