DataTalksClub / DataTalksClub/website

Wire approved messages through Relay and retire Datamailer

Open
#50 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

admin courses data-migration email enhancement events human integration operations P0 security testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Parent epic: #21. Coordination epic #6.

Resolved policy authority: #21, #22, #23, #124, and the owner decisions recorded in #227.

Normative authority:

  • _docs/specs/04-courses-and-cohorts.md, especially High-risk migration checks;
  • _docs/specs/05-events-registration-email.md, especially Course/cohort communication, Email preference and recipient semantics, Durable delivery model, and Relay sender and provider boundary;
  • _docs/specs/07-security-privacy-operations.md, for minimization, retention, redaction, suppression, and operational evidence;
  • _docs/specs/09-migration-rollout-roadmap.md, Milestones 5–8 and rollback;
  • _docs/specs/10-verification-strategy.md, Event and email plus deployed HUMAN gates; and
  • _docs/PROCESS.md.

PM disposition

GROOMED / P0 / DEPENDENCY-BLOCKED / HUMAN. Do not dispatch a #50 engineer lane.

#50 is the downstream integration and retirement coordinator. It consumes accepted business-trigger, recipient-policy, template, and delivery interfaces. It does not own Course/Cohort, Event, account, preference, or Slack business mutations; it does not define notification semantics; and it is not a prerequisite for implementing those non-sending domain services.

No issue, specification, local test, or source commit grants production Relay/provider/sender, credential, DNS, broad-recipient, Datamailer queue, or protected-data authority. The only approved sender configuration is the separately controlled development Relay sender ID courses, kept allowlisted/simulated except for one separately approved canary after every named safeguard passes. Production sender/domain activation and production Datamailer freeze/drain/retirement remain #74/infrastructure/operator work under separate explicit authority.

Outcome

Wire every implemented, specification-described Event, Course/Cohort, account, Slack, and marketing/newsletter purpose through the accepted website EmailDelivery and Relay boundary, while making all copied Datamailer send/requeue paths unreachable for new work. Preserve Datamailer only as send-disabled, read-only history/reconciliation input, and produce the deterministic inventory, import, freeze/drain, one-active-sender, rollback, and retirement evidence consumed by later authorized cutover.

Every business mutation and its one logical delivery intent plus durable job commit atomically in the owning domain service. A leased/fenced worker contacts Relay only after commit. #50 may register/adapt an already accepted trigger and routing contract; it may not recreate the mutation, preference decision, renderer, provider stack, or sender authority.

Exact dependency DAG

A dependency below means accepted, independently tested, PM-accepted, merged, pushed, and green with an immutable merge/source/API identity. An open issue, local candidate, issue prose, or synthetic test is not an accepted interface.

Relay and website delivery foundation
  Relay #1 -> website #48
  (Relay #2 + Relay #3 + #48 + accepted #31/#32/#33 foundations)
    -> #49 ordinary delivery
  #264 -> #284 -> #49 recovery-specific delivery

Event and recipient-policy lane
  #45 -> record the non-circular #46/#227 accountless-preference interface
  (#45 + #49 ordinary + recorded interface) -> #46
  (#49 ordinary + accepted #46 identity) -> #227 preference implementation
  (#46 + #49 ordinary + accepted management foundations) -> #47 domain operations/triggers
  (#46 + #47 + #227 + #48 + #49 ordinary) -> #50 Event/marketing routing

Course/member lane
  accepted #243 registration/conversion trigger
  + accepted #246 Enrollment command/removal trigger
  + accepted #55 homework/deadline/score trigger
  + accepted #56 project/peer-review/score trigger
  + accepted #58 certificate trigger
  + accepted #249 Slack grant trigger
  + #48 + #49 ordinary
  -> #50 Course/Slack routing

Purpose-specific decision lane
  #149 owner disposition and any accepted follow-up
  -> peer-assignment/score trigger semantics
  -> corresponding #50 routing only

Rehearsal and authorized operations
  accepted #48/#49 + accepted automated/source #50 candidate
  + accepted #60 send-disabled migration/freeze/outbox rehearsal
  + exact Relay deployment/OpenAPI/credential/callback/reconciliation/alarm/allowlist safeguards
  -> separately authorized one-recipient development `courses` canary
  -> #50 HUMAN acceptance
  -> #73 full development aggregate rehearsal
  -> #74-owned production cutover/freeze/drain/retirement
Dependency corrections
  • #21 is the parent/provider decision, not a circular implementation prerequisite. #22, #23, and #124 are accepted policy inputs, not open blockers.
  • Broad parent #54 is not a #50 dependency. #243 and #246 are the exact registration/Enrollment trigger inputs. Their existing child DAG remains authoritative.
  • #53 is transitive through #55/#56/#58 for Course/Cohort identity and lifecycle. #57 is transitive through #58 for completion/certificate eligibility. Neither is repeated as a blanket #50 blocker unless a future accepted purpose directly consumes a named symbol from it.
  • #55, #56, and #58 create only domain-trigger-ready events/scalar context. #50 is downstream and does not block their non-sending implementations.
  • #59 consumes accepted domain and management services and is not a #50 source prerequisite. It may expose accepted #48/#49 projections but cannot activate a sender or writable Datamailer fallback.
  • #60 rehearses course compatibility, migration, freeze, and outbox classification with all sending disabled. It neither activates nor closes #50, but its accepted evidence is required before the separately authorized development canary.
  • #73 is strictly downstream of #50 HUMAN acceptance. It runs the full development aggregate rehearsal with outbound paths disabled or contract-faithful fakes/zero-write dry-run. It cannot prove, authorize, or precede the live development canary.
  • #47's current blanket #49–#50 dependency is stale for its non-sending Event operations/trigger implementation. #47 may depend on #49 where it atomically creates logical intents, but #50 consumes the accepted #47 trigger contract. Do not create a #47 ↔ #50 cycle.
  • #227 defines recipient policy and Mailchimp marketing-only behavior; #50 consumes it. #50 is not a prerequisite for defining the preference subject/service. The unresolved #46/#227 accountless-subject/default/revision seam must be recorded before either implementation invents an identity or default.
  • #149 remains a real purpose-specific blocker. #22 structurally permits specification-described purposes, but it does not choose automatic versus explicit peer/score trigger, audience, or repeat/delta/resend behavior.
  • #284 is not a blocker for ordinary delivery or a source-only inventory. It is required before #49's recovery-specific implementation and before #50 can claim complete restore/rollback/retirement behavior.

Purpose ownership matrix

Purpose family Accepted business owner required before #50 routing #50 responsibility
Event verification and confirmation #46 plus #227 eligibility/preference interface Register immutable route/template/sender/context/idempotency mapping; no registration mutation
Event reschedule, cancellation, reminders, follow-up #47 over accepted #45/#46 state Adapt accepted audience/version trigger to #49; no Event lifecycle or bulk-operation reimplementation
Marketing/newsletter and unsubscribe enforcement #227 Consume canonical preference/eligibility service and accepted Mailchimp marketing-only state; no second contact/preference identity
Course registration confirmation #243 Route accepted registration/version event; no registration or Enrollment conversion logic
Enrollment welcome/removal #243/#246 as applicable Route accepted Enrollment/version event; no management command or state transition
Homework/deadline/score #55, plus #149 for any unresolved explicit notification semantics Route accepted immutable trigger/context only
Project/peer-review/score #56, plus #149 and any accepted follow-up Route accepted immutable trigger/context only
Certificate issue/revoke/reissue #58 Route accepted certificate/version event only
Slack access #249 Register the accepted slack_access route and secret-at-send handler boundary; no secret or grant ownership
Course updates A separately accepted named course-domain trigger is still missing No #50 implementation may infer publish/audience/version semantics
Account verification/password recovery A separately accepted accounts-owned trigger/token contract is still missing No #50 implementation may replace or wrap an unspecified authentication flow

A purpose with no accepted owner/trigger remains absent from #50 implementation. “Described in the specs” removes a separate owner-approval ceremony; it does not permit guessing a business mutation, audience, route identifier, template version, context schema, or idempotency version.

Source-only slice decision

One narrow source-only slice is valid in principle before the runtime dependencies: a deterministic, read-only inventory/manifest of the exact adopted Datamailer surface and a static completeness validator. It may classify source paths, callers, settings references, workers/commands, queues/outbox states, callback/audit/API/Studio routes, template/list identifiers, idempotency fields, external-ID fields, and retention owners against one immutable website/CMP source identity.

That slice must be filed and PM-groomed as its own child before engineering. It must be non-activating: no model/migration/runtime adapter, row or protected-data access, queue inspection, send disablement, importer, credential/provider/Relay call, sender configuration, or claim that work is frozen, drained, migrated, or retired. Its commit uses Refs #50; it cannot satisfy #50 acceptance.

No other source-only slice is currently valid:

  • a history importer or compatibility model needs accepted #49 storage/status/reconciliation interfaces and the exact retained legacy-output schema;
  • a preference or Mailchimp importer belongs behind #227's accepted canonical subject/apply service;
  • purpose wiring needs the accepted owning domain trigger listed above;
  • sender activation, queue drain, canary, and retirement are operational/HUMAN actions, never source-only evidence.

Implementation scope after dependencies pass

  • Register each accepted purpose with immutable Relay template key/version, sender/reply-to ID, typed secret-free context, logical idempotency/version inputs, classification, recipient-policy query, and owning service symbol.
  • Make every accepted domain trigger call #49's atomic logical-delivery/job service. No request transaction, model hook, callback, or migration contacts Relay.
  • Implement repeatable, send-disabled Datamailer history/mapping import only after its target interfaces are accepted. Preserve bounded provenance without rendered bodies, raw provider payloads, or unnecessary PII.
  • Classify every legacy item as terminal history, safely drained by the separately authorized old-system operator, explicitly cancelled, or send-disabled pending evidence requiring a later reviewed Relay resend. Import never sends or requeues.
  • Remove or hard-disable every new-send/requeue/immediate-dispatch path. Any retained adapter is read-only, owner/deadline/telemetry/removal-gate bounded, and cannot become a rollback sender.
  • Produce safe count/checksum/exception evidence and runbooks for hold/reconcile rollback, one-active-sender proof, observation, retention, and later credential/runtime removal.
  • Expose masked Studio/admin API trace from business action to website intent to Relay projection or legacy-history mapping through accepted #32/#33/#48/#49 services.

Acceptance criteria

  • Every implemented purpose is mapped exactly once to an accepted owner/trigger, audience/eligibility service, immutable template/version, sender/reply-to ID, typed context schema, logical idempotency/version inputs, classification/retention rule, and focused tests; missing-owner purposes remain absent and explicit.
  • Replay, recompute, resume, stale version, rollback, and concurrency produce one business outcome, one website intent, and one Relay message per intended purpose/version; changed work conflicts rather than reusing a key.
  • Recipient preferences are evaluated through #227's accepted service before intent creation and again where the accepted claim-time contract requires; suppression creates no unintended delivery/job/provider work and changes no unrelated category.
  • Repository/runtime inventory proves no new website path can call SES or Datamailer and identifies every legacy caller, worker, command, queue/state, callback, setting, API/Studio route, and removal owner.
  • History import is repeatable, source-pinned, send-disabled, and redacted; duplicate/missing/unknown IDs, count/checksum drift, and ambiguous pending work fail explicitly with zero queue/provider/callback side effect.
  • Every outstanding legacy item has one reviewed classification; no ambiguous item is silently sent, dropped, promoted, or duplicated.
  • Automated #50 evidence plus accepted #60 side-effect-disabled rehearsal prove migration/freeze/outbox classification, rollback hold/reconciliation, and one-active-sender invariants without treating rehearsal as live activation or #73 evidence.
  • Retained read-only compatibility scope, owner, metrics, deadline, removal gate, retention, and restored-backup behavior are documented and tested against accepted #284/#49 recovery semantics.
  • Studio/admin API trace and controls are service-parity, capability/object scoped, revision/idempotency protected, audited, masked, and free of bodies, full recipients, secrets, raw provider identifiers/payloads, or writable legacy controls.
  • Unknown/unimplemented purpose, unknown sender, incomplete route, broad recipient, absent credential/secret/alarm/reconciliation evidence, and every production configuration fail closed.
  • [HUMAN] Only after accepted #48/#49, the accepted automated/source #50 candidate, accepted #60 rehearsal, and exact credential/callback/reconciliation/alarm/allowlist safeguards exist, a separately authorized operator records one controlled one-recipient development courses canary using only redacted synthetic evidence. No #73 or production activation is inferred.
  • After the canary, PM records #50 HUMAN acceptance and one immutable redacted #50 acceptance identity that #73 may consume; #73 is not input to the canary or #50 acceptance.
  • Focused Django/migration/concurrency/security/redaction/contract/failure tests, graph-selected Playwright, inspected desktop/mobile screenshots, independent tester report, and automated/source PM acceptance pass on one frozen candidate before the HUMAN gate.

Required validation

  1. Exercise every accepted domain trigger across replay, changed version, stale preference, cross-object identity, suppression, rollback, concurrent request/job, worker restart, exact Relay replay/conflict, response loss, ambiguity, callback loss/reorder, and reconciliation.
  2. Run the source-pinned inventory and import twice with duplicate, missing, conflicting, unknown, drifted, and interrupted batches while all outbound mechanisms are disabled; compare exact safe counts/checksums.
  3. Rehearse a late legacy item during freeze, interrupted drain, restart, rollback, and restored backup; prove detection/classification and no fallback or dual sending.
  4. Verify no Datamailer/SES submit or requeue surface remains reachable from public views, Studio, admin API, commands, jobs, callbacks, or compatibility adapters.
  5. Through Studio/admin API, inspect masked queued/accepted/delivered/retryable/ambiguous/suppressed/dead/bounced/complained and legacy-history states with safe unavailable, stale, denied, and exception behavior.
Browser and screenshots

The independent tester exercises representative Event, course, account/Slack, and marketing purposes at desktop and mobile, including preference suppression, Relay unavailable/ambiguous state, imported legacy history, and migration exception. Screenshots must show the intended pages and contain no email, token, body, secret, raw context, provider payload/identifier, protected source value, or operator-only control leakage.

Lifecycle and commit convention

  1. Do not assign the broad #50 issue now.
  2. A separately groomed static inventory child may proceed independently and commits with Refs #50 after its own tester/PM/on-call lifecycle.
  3. After accepted foundations/triggers exist, PM freezes exact merge SHAs, Relay commit/OpenAPI/deployment identities, purpose matrix, source pin/schema, Datamailer inventory digest, #227 policy fingerprint, and #284/#49 recovery fingerprint for one isolated #50 source/runtime candidate.
  4. Automated engineering/testing uses synthetic data, fakes, allowlisting/simulation, and no provider or production access. After independent tester and automated/source PM acceptance, the source commit uses Refs #50 while the HUMAN criterion remains.
  5. Accepted #48/#49, the accepted automated/source #50 identity, accepted #60 send-disabled rehearsal, and exact Relay deployment/OpenAPI/credential/callback/reconciliation/alarm/allowlist safeguards form the complete canary entry gate. #73 is not part of it.
  6. Only a separately authorized operator may perform the bounded one-recipient development courses canary and post redacted evidence. PM then records #50 HUMAN acceptance.
  7. #73 consumes the immutable redacted #50 HUMAN acceptance identity for its later full development aggregate rehearsal. Only after accepted #73 may #74 coordinate separately authorized production cutover. Production sender/domain activation and Datamailer operational retirement remain #74/infrastructure authority; no earlier evidence is authorization.

Explicit non-goals

  • No direct website SES, new Datamailer send, writable compatibility adapter, local canonical template store/renderer, second provider-attempt/event stack, blind retry after ambiguity, dual sending, or fallback to Datamailer during rollback.
  • No invention or reimplementation of Event, Course/Cohort, registration, Enrollment, homework, project/review, score, certificate, account, Slack, preference, unsubscribe, or marketing business rules.
  • No inference of the unresolved #149 trigger/audience/repeat decision.
  • No wholesale retention of Datamailer models/bodies/PII, automatic requeue of imported work, destructive legacy deletion, or removal before retention/observation/rollback gates pass.
  • No production/protected-data/queue access, real recipient, provider/Relay/AWS/DNS/credential/sender mutation, deployment, canary, freeze, drain, or retirement without separate explicit authority.
  • No claim that #59 or #60 acceptance, #73 planning/rehearsal, a local green test, a source manifest, or elapsed time activates a purpose or retires a sender. #60 is a required pre-canary rehearsal input; #73 is a post-#50 aggregate consumer, never canary authority.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the named specifications, especially the migration, delivery, security, rollout, verification, and process documents, then inspect the adopted Datamailer surface described in the source-only slice. Before runtime work, produce a separate deterministic read-only inventory and completeness validator; implementation must wait for the listed accepted dependencies and must not activate sending or claim migration or retirement.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, infrastructure
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.