microsoft / microsoft/PromptKit

Pipeline quality gates: mandatory audit steps between pipeline stages

Open
#133 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
105
Forks
23
PR merge metrics
No merged PRs in 30d

Description

Summary

Add quality gates as a first-class pipeline concept — mandatory or recommended audit/verification steps that run between pipeline stages before the next stage can proceed.

Motivation

GitHub Spec Kit builds quality gates directly into its workflow:

  1. Spec quality checklist — After specification creation, validates completeness, testability, and ambiguity
  2. Constitution compliance — Before planning, checks that the spec doesn't violate project principles
  3. Pre-implementation checklist — Before coding, verifies all checklist items pass (stops and asks if failures exist)
  4. Consistency analysis — After task breakdown, validates spec ↔ plan ↔ tasks alignment with severity levels (CRITICAL > HIGH > MEDIUM > LOW)

These gates are mandatory, not opt-in, which is what makes them effective.

PromptKit already has the building blocks:

  • audit-traceability — validates cross-document alignment
  • audit-spec-alignment — checks spec ↔ implementation alignment
  • audit-code-compliance — checks code ↔ spec alignment
  • audit-test-compliance — checks test ↔ spec alignment

The gap: pipelines don't declare which audits should run between stages.

Proposed Design

  1. Pipeline schema extension: Add an optional gate field to pipeline stage transitions in manifest.yaml:

    pipelines:
      document-lifecycle:
        stages:
          - template: author-requirements-doc
            output_contract: requirements-document
          - gate: audit-traceability          # ← NEW: must pass before next stage
            severity: warning                 # or "blocking"
          - template: author-design-doc
            input_contract: requirements-document
            output_contract: design-document
          - gate: audit-spec-alignment
            severity: blocking
    
  2. Bootstrap engine integration: When running a pipeline interactively, the bootstrap engine would execute gate templates between stages and report pass/fail before proceeding

  3. CLI integration: npx promptkit assemble could emit a note about recommended gates when assembling a pipeline stage

Credit

This pattern is inspired by GitHub Spec Kit's built-in quality checkpoints. See their plan-template.md (constitution compliance gate) and checklist-template.md (pre-implementation quality validation) for the original implementation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pipeline definitions in manifest.yaml, then trace how the bootstrap engine runs stages and how the npx promptkit assemble CLI handles pipeline stages. Map where gate entries, severity, pass/fail behavior, and the recommended-gate note would be represented. Done means the schema, interactive execution, and CLI behavior consistently support the proposed mandatory or recommended gates.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.