DataTalksClub / DataTalksClub/website

Complete release-critical verification coverage and producer traceability

Open
#76 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

accessibility documentation epic integration operations P0 security testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Parent epic: #11

PM disposition

GROOMED / P0 / SOURCE-ONLY COORDINATION EPIC / DEPENDENCY-BLOCKED. Do not dispatch #76 as one broad engineering lane.

#76 completes the release-critical verification producer graph. It maps every applicable specification 10 release-critical scenario to an owning product/operations issue, a named automated test or explicitly owned HUMAN gate, and a versioned redacted evidence contract. It also coordinates bounded child/owning-issue work for every missing automated producer and proves that the complete graph is closed, deterministic, and current.

#76 does not generate the final release report (#77), run the full development rehearsal (#73), or authorize/execute production, provider, credential, protected-data, AWS, DNS/edge, Search Console, sender, Datamailer, or retirement actions (#74 and their owning HUMAN/provider gates).

Normative authority

  • _docs/PROCESS.md
  • _docs/specs/10-verification-strategy.md, especially release-critical scenarios, CI/test safety, and the acceptance-report inputs
  • _docs/specs/09-migration-rollout-roadmap.md, especially Milestones 6–8, rollback, and data-migration controls
  • _docs/specs/07-security-privacy-operations.md
  • the remaining numbered specifications and the accepted decision/inventory classification produced by #72
  • the current application-boundary and change-selective-verification contracts

Outcome and ownership boundary

The completed #76 artifact is the canonical verification-producer graph consumed by #77. Each row identifies:

  • normative specification requirement and accepted decision/inventory identity;
  • one owning implementation or HUMAN issue and accountable role;
  • named test, marker, manual procedure, or authorized-environment gate;
  • artifact type/schema, producer command or workflow, exact source/configuration/environment identity, digest/provenance, freshness/expiry, and redaction rules;
  • pass/fail/no-go semantics, allowed exception fields, and downstream #77/#73/#74 consumer.

#72 owns product inventories, decisions, and requirement-to-implementation classification. #76 consumes that accepted classification and owns requirement-to-verification-producer coverage. #77 consumes the frozen #76 graph and owns deterministic aggregation, checksums, exception validation, and go/no-go reporting. Missing behavior or tests remain with bounded domain/infra child issues; they are not silently implemented as one cross-domain #76 patch.

Accepted foundations and prerequisites

  • #75 is the accepted deterministic SQLite/Playwright harness and safety foundation; consume it rather than reimplement it.
  • #72 must provide the accepted current inventory/decision/ownership classification.
  • #63 supplies the residual security/authorization traceability producer after its #61/#32/#33/#52/#64/#66 handoffs. #141 remains the accepted non-identity security baseline and is not reimplemented here.
  • #64 supplies accepted privacy-rights, retention, erasure/propagation, processor-failure, and restored-backup tombstone producers through its owning children.
  • #65 supplies the automated accessibility producer and separately owns its real manual keyboard/screen-reader/zoom/reflow/contrast/motion/form evidence.
  • #66 supplies accepted observability, target/query, fault, backup/restore, RPO/RTO, alert, replay, and rollback producers through its owning children.
  • Every content, URL/SEO, Course/Cohort, account/Slack, Event/email, Studio/admin API, edge/infrastructure, migration, deployment, compatibility, and provider-bound producer referenced by the frozen graph must have an accepted owning contract.

A prerequisite is satisfied only by its current accepted source-side contract and immutable evidence envelope; an open parent issue, stale candidate, ordinary CI run, scheduled regression, synthetic result, or issue prose is not implicit acceptance. A HUMAN/provider row may remain pending for its owning downstream gate only when #76 records it explicitly as required/no-go with owner, authority boundary, environment, freshness, and consumer. It may never be reported as passed by #76.

Scope

  • Freeze a machine-readable and reviewable graph covering every applicable release-critical bullet in specification 10 without missing, duplicate, ambiguous, or unowned rows.
  • Reconcile the graph to #72's accepted ten-spec/current-decision inventory and fail on source requirement, owner, route/domain, test, artifact schema, or consumer drift.
  • Consume existing accepted automated producers and split every genuinely missing source-side behavior/test/artifact family into a bounded groomed owning issue.
  • Require deterministic negative/fault coverage for URL/SEO, content sync, Course/Cohort, account/Slack, Event/email/Relay, Studio/admin API, security/privacy/accessibility, edge/cache/WAF/cost/invalidation, deployment, backup/restore/rollback, and cross-provider degradation.
  • Require browser coverage of critical desktop/mobile, denial/error/degraded, accessibility, and screenshot states, with graph-selected routes and exact counts.
  • Define producer evidence for scheduled slow crawls, accessibility, dependency/security, restore, and deployed-smoke families, including complete failure enumeration and actionable redacted artifacts.
  • Validate that every skip, not-applicable result, reuse, exception, and manual gate has explicit schema-backed semantics; unknown, stale, partial, red, unowned, expired, or contaminated evidence yields no-go.

Explicit non-goals

  • No product/domain redesign, missing cross-domain behavior, or broad implementation bundle under #76.
  • No duplication of #72 inventories/decisions, #77 aggregation/report generation, #73 rehearsal execution, or #74 cutover/monitoring/retirement.
  • No live email/Slack/provider canary, credential or secret access, protected-source reconciliation, production-data access, AWS/Terraform apply, deployment, DNS/edge/indexing change, Datamailer freeze/drain, sender activation, legacy retirement, or other external mutation.
  • No inference that source-only automation proves a real screen reader, production identity/MFA, Relay/provider delivery, live edge/WAF/cache, restore/RPO/RTO, DNS, Search Console, production cutover, or observation window.
  • No verbal waiver, hidden skip, blanket flaky/pre-existing dismissal, stale evidence reuse, or exception without owner/risk/mitigation/expiry/approval.

Acceptance criteria

  • The frozen producer graph covers every applicable specification 10 release-critical scenario and reconciles to #72's accepted ten-spec/current-decision inventory with no missing, duplicate, ambiguous, or unowned row.
  • Every row names exactly one owning issue/role, test or HUMAN procedure, artifact schema/producer, immutable source/configuration/environment identity, digest/provenance, freshness/expiry, redaction rule, verdict semantics, and #77/#73/#74 consumer.
  • Every required source-side automated producer is accepted, merged, and reproducible from one current candidate envelope; URL/SEO/content/course/account/Event/email/Studio/API/security/privacy/accessibility/operations negative and fault scenarios emit deterministic evidence.
  • Browser producers cover graph-selected critical desktop/mobile and JavaScript/state variants, record exact counts, inspect required screenshots, and fail on unexpected error/debug/broken-layout, console, or network results.
  • Scheduled slow-suite producers enumerate all failures and publish actionable schema-valid artifacts; selector/reuse history cannot hide coverage, and missing/stale/mismatched evidence selects fresh verification or no-go.
  • Every manual, provider, protected-environment, or production-only row is explicitly marked required and pending/no-go with named owner, authority boundary, environment, artifact/freshness contract, and downstream gate; #76 does not execute or claim it.
  • Exceptions require owner, rationale, risk, mitigation, expiry/review date, approval, and release disposition; absent or expired fields fail validation.
  • An independent tester recomputes the frozen graph and source-side verification plan, validates every evidence envelope/digest and redaction boundary, and finds no unexplained skip, pending source-side producer, or graph drift.
  • PM confirms that #77 can deterministically consume the graph and that #73/#74 remain no-go whenever any required HUMAN/provider/production evidence is missing, stale, red, unowned, or unauthorized.

Required verification scenarios

  1. Remove, duplicate, mis-own, stale, or mismatch one row in each producer family and prove validation fails with complete diagnostics.
  2. Break one accepted invariant in each source-side category and prove its owning producer and #76 graph disposition fail without external side effects.
  3. Recompute the graph and plan twice from the same immutable inputs and prove stable ordering, identities, counts, and digests.
  4. Inject an unapproved skip, expired exception, foreign source/configuration identity, secret/email/provider payload, or incomplete manual record and prove publication/consumption fails closed.
  5. Validate representative desktop/mobile browser, screenshot, accessibility, fault, scheduled, restore-contract, and deployed-smoke artifact fixtures without contacting a shared or production environment.

Lifecycle and downstream DAG

accepted #72 inventory/decision classification
+ accepted #75 harness
+ accepted source-side domain/infra producers
+ #63 security + #64 privacy + #65 accessibility + #66 operations producer contracts
  -> #76 frozen verification-producer graph and source coverage
  -> #77 deterministic acceptance report/go-no-go consumer
  -> #73 separately authorized outbound-disabled development rehearsal
  -> #74 separately authorized production cutover/activation/observation/retention

PM/orchestrator first freezes the graph and grooms bounded missing producer issues. Each source change follows engineer (uncommitted) → independent tester, including graph-selected screenshots → PM acceptance → focused commit → local no-ff merge/push → on-call. #76 itself closes only after its source-side graph and producer criteria pass independently.

Outstanding HUMAN/provider/production gates remain open under their owning issues and remain required/no-go inputs to #77/#73/#74. human is therefore not a #76 lifecycle label unless #76 itself later acquires a manual criterion; no source-only completion grants external authority.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with _docs/PROCESS.md and _docs/specs/10-verification-strategy.md, then review the accepted #72 classification and #75 harness contract. Map each applicable release-critical scenario to its owning issue, producer or HUMAN gate, evidence schema, and downstream consumer without reimplementing child work. Done means the graph is deterministic, complete, validated, and ready for #77 while pending external gates remain no-go.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, python, sqlite
Domain
release, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.