DataTalksClub / DataTalksClub/website

Generate deterministic release acceptance reports and enforce go/no-go

Open
#77 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

data-migration documentation integration operations P0 security testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Parent epic: #11

PM disposition

GROOMED / P0 / SOURCE-ONLY RELEASE-REPORT CONSUMER / DEPENDENCY-BLOCKED. Do not dispatch engineering until #72 and #76 are accepted at frozen identities.

#77 implements one deterministic, schema-versioned acceptance-report generator and validator. It consumes the accepted #76 verification-producer graph plus provenance-bound evidence and derives an immutable report instance and fail-closed go/no-go verdict. It does not implement missing producer behavior, execute a rehearsal, contact an environment, or authorize a cutover.

A separate child is not needed for this source contract: the generator, schemas, validators, fixture corpus, review rendering, and consumer bindings form one bounded implementation. If a product-facing or hosted report UI is later requested, file and groom that separately.

Normative authority

  • _docs/PROCESS.md
  • _docs/specs/10-verification-strategy.md, especially the acceptance report, release-critical scenarios, test safety, and HUMAN gates
  • _docs/specs/09-migration-rollout-roadmap.md, especially Milestones 6–8, rollback, and migration controls
  • _docs/specs/07-security-privacy-operations.md
  • the accepted #72 ten-spec/current-decision classification
  • the accepted #76 verification-producer graph and evidence contracts
  • the accepted deterministic harness and change-selective verification contracts from #75/#104

The current decision index has 21 numbered sections, not the obsolete 18 in the former body. #77 must bind the exact accepted #72 decision-index identity/count and fail on drift rather than hard-code a permanently assumed count.

Ownership and dependency DAG

accepted #72 requirement/decision/inventory classification
+ accepted #75 deterministic local harness
+ accepted source-side domain/infra producer contracts
  -> accepted #76 verification-producer graph and coverage
  -> #77 deterministic aggregation/schema/checksum/go-no-go source contract
  -> #73 authorized outbound-disabled development rehearsal report instance + HUMAN receipt
  -> #74 separately authorized production cutover/final report instance + HUMAN receipt

#72 owns what must be traced. #76 owns which issue/role/test or HUMAN procedure produces each evidence row and the producer schema, identity, freshness, redaction, and verdict contract. #77 owns deterministic consumption and aggregation only. #73 and #74 own their separate execution environments, report instances, HUMAN decisions, and authority.

#75 is closed and is an accepted foundation, not an open work dependency. Domain/infra producers are transitive prerequisites through #76, not an unbounded duplicate direct dependency list on #77.

The #77 source implementation may validate synthetic fixture evidence for HUMAN/provider/deployed rows. It must represent missing real evidence as required and NO_GO; it must never perform or claim the real check. Actual #73/#74 execution still requires separate named authority and accepted dependencies.

Outcome

Provide one code-owned command/library that consumes only bounded machine-readable inputs and emits:

  • canonical JSON conforming to a versioned report schema;
  • a deterministic human-review rendering generated only from that JSON;
  • a SHA-256 checksum over canonical bytes;
  • complete diagnostics and a derived GO or NO_GO verdict; and
  • immutable consumer decision receipts that bind the exact report digest, phase, actor role, timestamp, decision, rationale, and authority reference without mutating the report they approve or reject.

Each report instance declares an explicit profile/phase such as development_rehearsal or production_cutover, references any required predecessor report digest, and includes exactly the rows required by the frozen graph for that profile. A later phase creates a new immutable report instance; it never edits an earlier checksum-bound report.

Required input contract

Every report binds:

  • report schema version, #72 classification digest/count, #76 graph schema/version/digest, and explicit evaluation timestamp;
  • application VERSION/full source SHA, construction timestamp, image digest, configuration digest, migration leaves, environment class, and deployed SHA/revision where the profile requires it;
  • active content source/release identities, commits, import counts, checksums, rejection/exception counts, and freshness;
  • URL/link/SEO and route/cache/private-bypass/invalidation/WAF/cost/allowance/TTL-rollback evidence;
  • Course/Cohort, account/member-profile/registration/Slack, Event/email/Relay, and migration reconciliation;
  • Studio/admin API capability and OpenAPI parity;
  • unit/integration/contract/browser/security/privacy/accessibility/scheduled-suite evidence;
  • Terraform/plan/deploy/readiness/smoke/backup/restore/image-rollback/SLO/fault evidence;
  • every automated, HUMAN, provider, protected-environment, and production-only graph row with its owning issue/role, status, artifact identity, provenance, source/configuration/environment identity, timestamps, freshness/expiry, redaction verdict, and downstream consumer; and
  • structured exceptions and separately checksummed consumer decision receipts.

Inputs are local artifact files or fixture data only. The generator performs no test, migration, browser, network, deployment, provider, protected-data, credential, AWS, DNS, Search Console, email, sender, Datamailer, or production action.

Determinism and go/no-go rules

  • The same canonical inputs, explicit evaluation timestamp, schema, policy, and graph produce byte-identical JSON, review output, ordering, diagnostics, and checksum. No implicit wall clock, filesystem order, mutable ref, network lookup, or ambient environment changes output.
  • GO requires every row required by the selected profile to be present exactly once, schema-valid, green or explicitly satisfied by its allowed evidence state, fresh, redaction-clean, identity-consistent, and owned.
  • Missing, red, stale, future-dated, duplicate, ambiguous, unowned, unauthorized, contaminated, unknown-schema, wrong-profile, wrong-source/config/environment, checksum-mismatched, or incomplete evidence yields NO_GO.
  • skipped, reused, and not_applicable remain explicit. Reuse requires its graph-authorized envelope and digest; an unexplained skip or applicability mismatch yields NO_GO.
  • Required HUMAN/provider/protected-environment/production evidence that is absent or pending yields NO_GO. Source-only fixture validation cannot promote it to passed.
  • An exception requires stable ID, affected graph rows, owner, rationale, risk, mitigation, issue/approval reference, approval role and timestamp, review/expiry date, and release disposition. Missing, conflicting, unapproved, out-of-scope, or expired exceptions yields NO_GO; an exception cannot erase the underlying result.
  • Secret, token, credential, email/profile/registration value, provider body/identifier, raw protected-source locator, reversible identity digest, or other prohibited canary blocks publication. Diagnostics and review output remain redacted and bounded.
  • A HUMAN GO receipt cannot override a machine NO_GO. Unknown or mismatched report digests, roles, phases, timestamps, authority references, or predecessor digests invalidate the receipt.

Scope

  • Define the versioned input, report, exception, diagnostic, and decision-receipt schemas.
  • Implement canonical parsing, validation, aggregation, stable sorting/serialization, checksum generation, and human-review rendering.
  • Consume the accepted #76 graph without duplicating or weakening its producer contracts.
  • Reconcile every report row to the frozen #72 ten-spec/current-decision index and fail on missing, duplicate, ambiguous, stale, or newly unclassified requirements.
  • Add a project-local synthetic fixture corpus for complete-green and each fail-closed class.
  • Add bounded Make/CI validation that proves schemas, fixtures, determinism, redaction, and consumer bindings without contacting an external system.
  • Define the immutable handoff contract used by #73 and #74, including profile/predecessor binding and separate HUMAN decision receipts.

Explicit non-goals

  • No missing product/domain/infra producer implementation, broad cross-domain suite work, or duplication of #72/#76.
  • No execution of #73 rehearsal or #74 production cutover/activation/observation/retention.
  • No live browser/environment check, production-like data read, protected source, credential, provider, AWS/Terraform apply, deployment, DNS/edge/indexing change, Relay/email/Slack canary, Datamailer action, sender activation, or legacy retirement.
  • No product UI. A generated static review artifact is internal, contains no scripts or remote assets, and is not a hosted route. Any hosted/product-facing UI is a separately groomed issue.
  • No hand-edited verdict, hidden skip, verbal waiver, mutable report, stale artifact reuse, or approval that overrides a failed gate.

Acceptance criteria

  • Versioned schemas cover report inputs, canonical report output, exceptions, diagnostics, phase/predecessor identity, and separate HUMAN decision receipts.
  • Traceability reconciles exactly to the accepted #72 ten-spec/current-decision classification and accepted #76 graph; on the current index this is ten numbered specs and 21 decision sections, with drift failing closed.
  • Every required acceptance-report family in specification 10 is represented with explicit counts, statuses, skips/reuse/applicability, owner, environment, immutable identity, timestamps, provenance/digest, freshness, redaction, and consumer.
  • Canonical JSON, human-review output, diagnostics, and SHA-256 are byte-identical for identical explicit inputs; changed semantic input changes the checksum.
  • GO is impossible when any selected-profile row or required artifact is missing, red, stale, duplicate, ambiguous, unowned, unauthorized, contaminated, mismatched, unknown, or incomplete.
  • HUMAN/provider/protected-environment/production rows remain explicit required NO_GO inputs until valid owning evidence exists; no local fixture or exception silently satisfies them.
  • Exceptions enforce owner, affected rows, rationale, risk, mitigation, approval reference/role/timestamp, review/expiry, and disposition; invalid or expired exceptions block.
  • Redaction/publication validation rejects prohibited canaries from inputs, canonical output, diagnostics, review rendering, and receipts without echoing the value.
  • #73 can create an immutable development-rehearsal report instance and bind a separate HUMAN receipt; #74 can consume that exact digest and create later immutable production/final instances and receipts without mutating prior reports.
  • Focused source-only verification passes under the accepted #75 harness and #104 verification-plan contract; an independent tester recomputes the plan/digests and finds no unexplained skip or external side effect.

Required verification scenarios

  1. Generate the same complete-green synthetic profile twice from shuffled filesystem/input ordering and prove byte-identical canonical JSON, review output, diagnostics, and checksum.
  2. Remove, duplicate, stale, future-date, corrupt, re-own, or identity-mismatch one row in each producer family and prove complete deterministic NO_GO diagnostics.
  3. Exercise red, skipped, reused, not-applicable, pending HUMAN/provider, wrong-profile, unknown-schema, and predecessor/report-digest mismatch states and prove none can be silently promoted.
  4. Add valid, unowned, unapproved, conflicting, out-of-scope, and expired exceptions; prove only schema- and graph-authorized dispositions are represented and no exception hides the underlying evidence state.
  5. Inject representative secret, token, credential, email/profile/registration value, provider body/identifier, protected-source locator, and reversible digest canaries at every input/output/diagnostic/receipt boundary; prove publication fails without echo.
  6. Create #73-style and #74-style synthetic report/receipt chains; prove phase, predecessor, immutable identity, authority, and machine-verdict checks reject reordering, mutation, replay, or a HUMAN GO over machine NO_GO.
  7. Change the frozen #72 decision index or #76 graph without updating the report contract and prove drift fails with actionable bounded diagnostics.

Browser and screenshot disposition

No product route or browser rendering changes. The generated review artifact is source-only/internal and verified byte-for-byte plus accessibility/static-markup checks if HTML is chosen. Under the current graph, Playwright screenshots are not_applicable; if implementation adds a hosted or browser-rendered surface, stop and re-groom that work as a separate issue with desktop/mobile tester screenshots.

Lifecycle and activation boundary

After #72 and #76 are accepted at exact identities, engineering may create one fresh isolated #77 worktree from current main, implement the bounded source contract uncommitted, generate the required change-selective verification plan, and hand it to an independent tester. PM acceptance, focused commit with Closes #77, local --no-ff merge/push, and on-call follow only after the normal gates pass.

Closing #77 proves only that the deterministic report consumer is accepted. It does not prove a release candidate is ready. #73 remains responsible for the separately authorized development rehearsal and its report/receipt; #74 remains responsible for separately authorized production execution, final evidence/receipts, observation, and retention. No source issue, report schema, green fixture, closed dependency, or PM acceptance grants external authority.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read _docs/PROCESS.md and the cited specifications first, then inspect the accepted #72 classification, #75 harness, #76 producer graph, and #104 verification-plan contract. Define the bounded source-only generator, schemas, fixtures, and Make/CI validation described here; done means deterministic reports and rendering, fail-closed diagnostics and verdicts, redaction checks, and immutable consumer bindings without external actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
json, python
Domain
release, testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.