Reconcile the privacy authority, inventory, notice, and retention registers

Parent epic: #64
Owner decision: #23 (closed)

Product outcome

The repository has one canonical, versioned, machine-reviewable privacy authority register that reconciles the current source tree, _docs/legal/data-flow-inventory.md, specification 07, the public legal-page candidates, and closed #23 without treating observed implementation as legal approval. Every current or explicitly planned personal-data flow is traceable to an owning domain, source object/surface, processor boundary, policy status, retention/right disposition, evidence, owner, and freshness rule. Unknown legal values remain explicit, validated HUMAN blockers.

This issue has two gates. The bounded source-only register/reconciliation slice is implementation-ready now. Public legal copy and final closure remain HUMAN/decision gated. The source-only slice is committed with Refs #254; #254 stays open until the authorized legal/privacy review and rendered-page acceptance below are recorded.

Normative authority

• _docs/PROCESS.md: role-separated lifecycle, versioned verification plan, sensitive-artifact handling, screenshots, and Refs convention for remaining HUMAN work.
• _docs/specs/07-security-privacy-operations.md: privacy baseline, rights, approved default retention periods, erasure/propagation, redaction, and restore requirements.
• _docs/specs/open-decisions.md, resolved decision 12, and closed #23: privacy contact; no age gate or differential minors treatment; full-erasure default; approved spec-07 default periods; narrow issued real-name certificate exception.
• _docs/specs/01-platform-architecture.md and _docs/architecture/app-boundaries.md: one deployment, domain ownership, services/adapters, and no cross-domain mutation.
• _docs/specs/04-courses-and-cohorts.md, 05-events-registration-email.md, 06-studio-and-admin-api.md, 09-migration-rollout-roadmap.md, and 10-verification-strategy.md: domain records, processor boundaries, authorization, rollout, and evidence.
• _docs/legal/data-flow-inventory.md and closed #125: current source-reviewed inventory/legal-page baseline, not approval of missing legal facts.
• Parent #64 and dependencies #49, #66, #251: target email, restore/tombstone, and accounts-adapter contracts remain separately owned.

Closed #23 facts that must be recorded exactly

1. Privacy contact: Alexey Grigorev, alexey@datatalks.club.
2. No age verification, age gate, or differential minors treatment is approved; do not claim the service has no children or that a jurisdictional age fact was verified.
3. A learner privacy request defaults to full erasure of personal data, not generic anonymize-and-retain.
4. Spec-07 defaults are approved: unverified registrations/abuse metadata 14 days; event registration PII 90 days after the event; website logical delivery intent/redacted Relay projection metadata 180 days; security/audit events one year; development logs 30 days. Relay-owned suppression and the post-active educational-record schedule are not thereby fixed.
5. Ordinary leaderboard/certificate presentation defaults to a generated anonymous name. A real name explicitly chosen for an already-issued certificate remains on that issued certificate; this is the only accepted educational-record exception and must not become a general retention escape hatch.

Remaining HUMAN/legal authority

Do not infer or mark accepted:

• controller/operator legal identity, address, representative, tax/registration details, competent supervisory-authority/complaint wording, governing law, venue, statutory wording, or public contact details other than the approved privacy contact;
• controller/processor/subprocessor roles, regions, international-transfer mechanism/safeguards, contracts, incident contacts, or review/expiry dates;
• per-field/integration purpose and lawful basis, publication basis, legitimate-interest assessment, legal/security/abuse holds, or restriction/objection semantics;
• final notice/consent text, version, legacy-evidence migration rule, or displayed effective/last-updated date;
• the exact end of an “active” educational record, its post-active schedule, production-log period, or the implementation boundary of the issued-certificate exception;
• accountless identity proof, recovery, appeal, escalation, deadline, and requester-evidence rules;
• processor receipt/confirmation, retry/escalation deadline, suppression treatment, tombstone expiry, or legal-hold interaction.

Every such field uses a structured human_required state, names the decision owner, and links the blocking authority item. Empty strings, prose such as “TBD”, guessed values, and copying another product are invalid.

Canonical files and ownership

The source-only slice owns exactly:

• _docs/legal/privacy-authority-register.json — the sole hand-edited canonical register;
• _docs/legal/privacy-authority-register.schema.json — JSON Schema Draft 2020-12 contract;
• _docs/legal/data-flow-inventory.md — deterministic rendered projection of the canonical JSON, headed with register version, source commit/digest, render command, and HUMAN warning;
• scripts/validate_privacy_authority_register.py — fail-closed semantic/source-reference validator;
• scripts/render_privacy_authority_register.py — deterministic renderer with --check;
• scripts/tests/test_privacy_authority_register.py — schema, validator, renderer, redaction, and stale-reference tests.

No second hand-maintained CSV/YAML/Markdown authority is introduced. Schema and renderer code may contain enums/formatting only; they may not contain a shadow copy of policy rows.

Canonical register contract

Top-level required fields:

• schema_version (integer 1), register_version (date-based immutable version such as 2026-08-30.1), inventory_as_of_commit (full Git SHA), inventory_as_of_date (ISO date), authorities, decision_owners, processors, human_gates, model_coverage, and flows.
• Authority records have unique stable IDs, type (spec, issue_decision, source_observation, or human_decision), exact repository path or GitHub URL, state, and version/date where known.
• HUMAN gates have unique IDs, exact question, accountable role, state human_required, affected flow/field IDs, and no fabricated due date.

Every flows[] row has a stable slug ID and these required sections:

• inventory: current, legacy_current, planned_absent, aggregate_only, public_source, or inactive; subject classes; data classes; domain owner; and exact source references (kind, repository path, and model label/route name/symbol/setting/template where applicable).
• purpose: one or more purpose records with status, exact purpose, lawful-basis status/value, and authority IDs. Unknown basis is human_required, never consent, contract, legitimate_interest, or legal_obligation by inference.
• parties: controller status/reference plus processor/subprocessor IDs, role status, region status, transfer status, and safeguards status. A service named in source is not automatically an approved processor.
• notice_consent: notice ID/version/status/source route; whether evidence is transactional/privacy/marketing/analytics/publication; withdrawal behavior status; and legacy provenance. accepted_newsletter and other historical combined/boolean evidence are explicitly legacy_unversioned and can never satisfy current affirmative marketing consent.
• retention: trigger, duration/schedule, hold status, disposition, policy status/authority, propagation status, and exception IDs. Approved durations use machine values (P14D, P90D, P180D, P365D, P30D); the educational-record end trigger remains human_required.
• rights: access, portability, correction, erasure, restriction, objection, withdrawal, and publication-removal dispositions, each supported, planned, not_applicable, exception, or human_required, with owning adapter/service or blocking issue.
• implementation: current service/adapter, target adapter issue, exposed routes/exports/public projections, and implemented, legacy, planned_absent, or not_applicable.
• verification: focused test IDs/paths, redacted artifact type, and evidence status; absence assertions are first-class evidence.
• governance: accountable product/domain role, policy owner, effective-date status, review-date/status, freshness inputs, and exceptions.

processors[] uses stable IDs and separately records observed/configured/target status, controller/processor role status, data classes, source references, region/transfer/safeguard/contract/incident-contact states, and owner/review state. GitHub, Google, Slack, AWS, legacy Datamailer, historical Luma/Eventbrite inputs, and target Relay must not be collapsed into one generic “third parties” row.

model_coverage[] classifies every concrete first-party Django model exactly once as personal, pseudonymous, credential, public_personal, aggregate_only, or non_personal_operational, links it to one or more flow IDs, and gives a reason for non-personal/aggregate classification. Framework/allauth models used by authentication are explicitly referenced through the auth flow rather than silently omitted.

Required current-source inventory

At minimum, reconciliation at origin/main must contain separate flow groups for:

1. accounts identity/authentication/social-provider links, tokens/sessions, current CustomUser profile fields, identity aliases/quarantine/reconciliation evidence, and the absence of runtime MemberProfile, SlackAccessGrant, and privacy-request models;
2. course campaign registration and the legacy combined accepted_newsletter boolean, including form/service/API/export surfaces;
3. course enrollment, submissions/answers/projects/votes/peer reviews/scores/complaints/certificate fields and the separate public leaderboard/certificate/profile projections;
4. native Event Q&A question text, optional author name, participant/rate-limit digests, co-host credentials, retention field, and public/moderation surfaces;
5. historical Eventbrite/Luma aggregate import provenance and protected mapping identifiers, with an explicit assertion that no attendee identity/consent payload is stored by that aggregate flow and that native event registration is absent;
6. legacy Datamailer contact events, outbox payloads/errors/responses, send audits, callbacks, commands, and API exposure; separately record that email_app/Relay delivery models are planned/absent and #49 owns the target;
7. public GitHub-authored people/contribution/source provenance and correction/publication boundary;
8. security/audit events, jobs, staff sessions, API principals/credentials/rate admissions, exports, logs, and operational evidence, distinguishing credentials/pseudonymous IDs from ordinary metadata;
9. the dtc_analytics_consent preference cookie and executable absence of an analytics provider/measurement path;
10. the transient CloudFront country suggestion and its no-store/no-log boundary;
11. configured/current hosting and integration boundaries (AWS, GitHub, Google, Slack) and each processor/region/transfer HUMAN gap;
12. the approved #23 policy facts and every current public Privacy/Terms/Impressum contradiction or missing value as a named legal-copy mismatch/HUMAN gate.

The engineer expands this list when source inspection finds another current personal-data surface. It may split a group when purposes, rights, retention, or processors differ; it may not merge unlike policy rows merely to reduce row count.

Validation and generated checks

The validator and tests must fail on:

• schema errors; duplicate flow/processor/gate IDs; missing required sections; noncanonical ordering; or nondeterministic renderer output;
• a concrete first-party model missing from model_coverage, duplicate model classification, a flow referring to an unknown model, a named route that does not resolve to the recorded view, a missing file/template/symbol/setting, or an undeclared processor reference;
• an inventory_as_of_commit that does not match the verified source-reference digest/freshness inputs; renderer drift; or a changed referenced source without register review;
• an accepted policy value without an authority ID, a HUMAN value represented as accepted, a planned/absent surface represented as current, or a source observation represented as legal approval;
• unversioned/combined legacy evidence represented as current marketing consent; a consent-required purpose with no version/withdrawal state; or an inactive analytics/Relay path represented as active;
• retention durations that contradict the exact #23 defaults; educational retention treated as fully decided; generic anonymize-and-retain; or any retention exception other than the narrow issued-real-name-certificate record unless a new HUMAN authority exists;
• missing right/adapter/test/freshness dispositions, unresolved conflicts hidden by empty/null/free-form placeholders, or a public legal mismatch not linked to an open HUMAN gate;
• real participant data, tokens, secrets, raw provider payloads, message bodies, production exports, or reversible identifiers in the register, generated document, fixtures, logs, or test output. Synthetic .example/.invalid values and the explicitly approved privacy contact are the only email literals allowed.

The renderer is byte-for-byte deterministic and --check never writes. Validation performs no network/provider/production/database mutation and inspects only source/configuration metadata.

Scope now: source-only reconciliation

• Add the canonical register, schema, deterministic projection, validator, and tests above.
• Inventory source paths/models/routes/settings and processor names without reading protected or production data.
• Record #23 exactly and enumerate every unresolved HUMAN item/conflict.
• Replace the stale hand-maintained prose inventory with the generated projection.
• Produce no runtime behavior, database migration, provider request, production setting, policy decision, or legal advice.

Public legal-page gate

The source-only slice does not change templates/public/legal/privacy.html, terms.html, impressum.html, their displayed “Last updated” date, sitemap/indexability, or runtime views. The register records their current statements and contradictions.

Only after an authorized owner/legal reviewer supplies the exact approved wording and displayed effective date may a follow-up phase in this same issue update public copy. That phase may incorporate the five #23 facts but must retain every still-applicable owner/legal warning; it cannot present draft rows as effective policy. Terms/Impressum operator, statutory, governing-law, transfer, and complaint text remain blocked until specifically approved.

Dependencies and non-goals

• #23 is resolved only for the five facts listed above; it is not blanket legal approval.
• #49 owns Relay delivery/processor runtime; #66 owns restore rehearsal; #251 and other domain-adapter issues own domain export/erasure behavior. #254 inventories them but does not implement or unblock their runtime by assertion.
• No runtime privacy request, accountless proof, export, correction, retention runner, erasure, tombstone, processor receipt, cache/search invalidation, Studio/admin surface, domain adapter, production import, or production execution.
• No real requester/provider/AWS/GitHub/Google/Slack/Datamailer/Relay action; no contract/legal inference; no production-data inspection.
• No model/migration/API/OpenAPI/auth/permission/service/template/JavaScript/CSS change in the source-only slice.
• No copied AI Shipping Labs operator/legal values and no claim that source tests constitute legal acceptance.

Acceptance criteria — source-only engineering slice

• The canonical JSON, Draft-2020-12 schema, validator, deterministic generated Markdown, and focused tests exist at the exact paths above, with no shadow authority.
• The register records the five #23 facts exactly and represents every remaining legal value as a linked human_required gate.
• Every concrete first-party model is classified exactly once; every personal/credential/pseudonymous/public-personal flow links exact current models/routes/files/settings/processors, ownership, policy status, rights, adapter, test/artifact, and freshness.
• All required current-source groups and negative/absent boundaries above are represented, including legacy Datamailer versus absent target Relay, legacy newsletter evidence, aggregate-only historical events, no analytics provider, and public learner outputs.
• Validation rejects missing/duplicate/stale/unknown references, policy-status elevation, consent laundering, retention/erasure conflicts, hidden placeholders, undeclared processors, and sensitive evidence.
• _docs/legal/data-flow-inventory.md is reproducible byte-for-byte from the JSON; --check passes on a clean tree and the generated header identifies version/source/digest/HUMAN status.
• Focused unit tests, validator/renderer checks, repository quality checks, migration drift check, and the graph-selected versioned verification plan pass using uv/Make.
• Source-only screenshots are not_applicable: the verification graph proves no template/static/view/render impact, and the tester records that disposition rather than capturing unrelated pages.
• Engineer and independent tester evidence use synthetic values/counts/digests only and contain no protected data or secret/provider payload.

Acceptance criteria — HUMAN/legal closure phase

• [HUMAN] Authorized legal/privacy owner resolves every gate required for public policy: operator/controller details, processor/region/transfer/safeguard register, per-field purpose/basis, notice/consent versions and legacy mapping, educational/production-log/hold schedules, rights/appeal rules, processor receipts/tombstone expiry, exact legal wording, and displayed effective date.
• [HUMAN] The accepted authority response names the register version and says ACCEPT, ACCEPT WITH CHANGE: <exact changes>, or REJECT: <reason>; PM records it without interpreting silence as approval.
• If public templates change, focused legal Django/link/indexability/accessibility tests and the graph-selected Playwright tier pass; an independent tester captures and inspects /privacy, /terms, and /impressum at desktop 1440×900 and mobile 390×844 in the affected themes/states under .tmp/screenshots/issue-254/, including unresolved-warning visibility, heading/link integrity, 200% reflow, keyboard access, no overflow, and no error/debug page.
• [HUMAN] Authorized reviewer confirms the rendered copy, contact, warnings, rights/retention/transfer wording, and displayed effective date match the accepted register version. Only then may PM accept closure and the final focused commit use Closes #254.

Required verification commands

Source-only engineer/tester minimum (plus the generated change-selective plan):

uv run --frozen python scripts/validate_privacy_authority_register.py
uv run --frozen python scripts/render_privacy_authority_register.py --check
uv run --frozen python -m unittest scripts.tests.test_privacy_authority_register
make migrations-check
make verification-quality

If legal templates later change, add focused content.tests.test_legal_pages, the graph-selected Playwright tier including playwright_tests/test_legal_footer_125.py, and independent rendered screenshots. No screenshot can satisfy the HUMAN legal decision itself.

Delivery convention

After independent tester pass and PM acceptance of the source-only slice, the engineer commits only that accepted slice with Refs #254; retain human and decision, leave #254 open, and do not push/merge outside the normal orchestrator lifecycle. Final closure requires the separate legal-copy/HUMAN gates above and a later focused Closes #254 commit after the complete lifecycle.

Start with _docs/PROCESS.md and _docs/specs/07-security-privacy-operations.md, then inspect the listed canonical register, schema, validator, renderer, and focused tests. Run scripts/tests/test_privacy_authority_register.py and the renderer check before reconciling the source tree and required flow groups. Done means deterministic validation and rendering pass, source references are complete, and unresolved legal values remain explicit human_required gates.