DataTalksClub / DataTalksClub/dataops

Automate reviewed sponsor communications from booking milestones

Open
#114 30 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend data enhancement frontend human infra P1 portal testing
Dominant language
TypeScript
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Automate reviewed sponsor communications from booking milestones

Status: blocked overall — the permanent product source is accepted, published, and proven in a successful default-off production deployment; private HUMAN readiness review is eligible now, while any configuration-changing enablement or controlled send waits for #166 cleanup and authorized canonical Sponsor Booking/contact data
Tags: enhancement, portal, frontend, backend, data, infra, testing, P1, human
Depends on: #166 Phase D, canonical first-write evidence, and removal of the temporary cutover workflow before any enablement deploy; a communications owner; later one authorized canonical Sponsor Booking with an active contact; private template/HMAC/SES configuration and a DataTalks.Club-owned test inbox
Satisfied dependencies: #111, #129, and the #136/#143/#147 Sponsor table chain are complete; #174 removed the migration-only communication machinery
Not a dependency: #113 finance enablement; the closed Sponsor migration framework; the retained one-off Sponsor source importer
Blocks: none
Next owner: Alexey or Valeria may complete Stage A privately now; On-Call/credentialed operator owns Stage B only after #166 cleanup; an authorized admin owns Stages C-D
Resume condition: Stage A records the private product decisions; #166 records accepted Phase D, canonical first-write, and cleanup evidence; one ordinary canonical Booking/contact and the owned inbox/provider prerequisites exist
Current evidence: integration TEST PASS, PM acceptance, default-off deploy/production smoke, migration cleanup, and final empty Sponsor table state

Outcome

Sponsor booking milestones produce review suggestions. A person authors the exact message, and a currently authorized admin explicitly approves the exact immutable preview before one asynchronous email attempt. Nothing automatically drafts, approves, sends, retries an ambiguous effect, or targets a real sponsor during automated verification.

The implementation was accepted at source commit 949165abe69a4dddfc701b5883c45355c15f5f86 and composed with Sponsor Finance on main through merge 3365c2abe31d3d4da285b1bb1853c79c3bb3fc65. The successfully deployed 794076354048f6ff3417d55d13d334caf4237cea contains that source in ancestry. Normal CI run 31711997388 proved OIDC deployment, runtime seed, and production smoke with SponsorCommunicationSendEnabled=false.

That default-off proof remains valid. #166 does not require this feature to be reimplemented, but its protected A/B/C/D history must not carry a new flag, secret, provider, seed, or feature change. Any enablement change waits until #166 restores the final Tasks table and ordinary push deployment, proves canonical writes, removes the temporary cutover machinery, and is accepted On-Call.

Permanent product boundary

  • The four booking occurrences create deterministic, deduplicated suggestions only: booking confirmation, materials reminder, publication live, and performance follow-up. Repeated or out-of-order evaluation cannot create a draft, approval, attempt, or provider effect.
  • An authenticated operator or admin creates/edits an immutable draft version. A currently active admin approves the exact current preview. Approval atomically queues work and never calls SES inline.
  • One attempt sends one plain-text UTF-8 message to one canonical active contact address through the configured SES identity/configuration set. There is no HTML, attachment, CC/BCC, bulk send, campaign, or caller-supplied executable recipient.
  • The provider client performs one attempt. Accepted means provider-accepted, not delivered. Any ambiguous post-dispatch result becomes outcome_unknown and is never automatically resent. Admin reconciliation of an uncertain real-world effect remains a permanent live-product safety action.
  • Immutable sanitized provider facts derive the visible state. Bounce/complaint and manual suppression remain permanent product safety behavior. Private content, addresses, tokens, HMAC material, and raw provider diagnostics do not enter public export, logs, failure artifacts, or this public issue.
  • The deployment and table config default sending off. Disabling blocks new approval/pre-dispatch work while preserving history and provider facts. Re-enabling requires a fresh exact preview and approval.
  • Separate versioned private template and HMAC secrets provide content and suppression lookup material. The public repository contains only the schemas, resource references, synthetic fixtures, and safety code.

The accepted implementation/test detail remains in the linked frozen Tester and PM evidence. No new source implementation stage is open on this issue.

Migration boundary after #174

Sponsor Communications migration-only support is deleted and must not return:

  • no suppression migration endpoint;
  • no suppression-orphan list/reconcile endpoint or record type;
  • no scan/checkpoint/resume/rollback/backfill planner;
  • no migration UI controls or ongoing migration test suite.

The permanent admin route for an actual outcome_unknown send attempt is not migration support: it resolves uncertainty about a live external effect without sending again. Ordinary suppression enforcement/removal and immutable provider-event reconciliation are likewise permanent product behavior.

The Sponsor table is currently CloudFormation-owned, ACTIVE, empty, and has all four final active indexes. Initial enablement therefore uses one active HMAC write/read version and requires zero uncovered live suppressions; there is no historical suppression migration to run. If a future key change encounters old live suppressions, sending stays fail-closed and a separately groomed permanent maintenance decision is required—this issue does not authorize a one-off migration API.

The retained backend/scripts/import-sponsor-crm.ts is an out-of-runtime, one-off raw-source importer. It is not an #114 dependency and this issue never authorizes running it. Stage C needs one ordinary authorized Booking/contact. If that data is unavailable, the send stays blocked until it is created through normal product use or a separately approved future import window.

Staged HUMAN release plan

Stage A — private communications decision (eligible now)

Owner: Alexey or Valeria. No AWS/provider/data action is needed.

Record privately:

  • the responsible communications owner and allowed operator/admin roles;
  • the four reviewed template intents and approved placeholders;
  • the owned test inbox, sender identity, optional reply-to, and acceptable real-world test window;
  • suppression/complaint handling responsibility and the rule that an ambiguous attempt is never blindly retried;
  • the exact enable/disable decision and evidence owner.

Acceptance is a private decision record with no template text, address, contact, sponsor, secret, or provider detail copied into this public issue.

Stage B — default-off deployed readiness and enablement

Owner: credentialed operator and On-Call. Start only after #166 cleanup restores ordinary push deploy.

  1. Prove the ordinary deployed artifact contains accepted #114 source and starts with sending disabled; history may remain readable, but approval and dispatch must fail closed.
  2. Create/version-check the separate private template and HMAC secrets, using one active HMAC version for the empty table. Verify the exact SES identity, sender, configuration set, sanitized EventBridge projection/destinations, and least-privilege effective IAM.
  3. Confirm secrets, private content, addresses, and raw provider events do not appear in logs, workflow output, public export, or public evidence.
  4. Supply the reviewed deployment variables and enable only through the restored ordinary main CI/CD path. Do not manually deploy the app and do not modify or dispatch a #166 phase commit for this feature.
  5. Verify the enabled portal state and that no email is sent before an admin approves a fresh exact preview.

Sanitized evidence may record resource state, counts, configuration generations/digests, workflow run, and pass/fail results only.

Stage C — one controlled owned-inbox send

Owner: authorized admin. Requires Stage B plus one canonical Booking with an active contact whose address is the pre-approved DataTalks.Club-owned inbox.

  1. Produce the eligible suggestion through the ordinary milestone evaluator and prove duplicate evaluation produces the same suggestion with no draft/send side effect.
  2. Author a fresh version, inspect the exact preview, and approve once as the current admin.
  3. Observe exactly one provider-accepted attempt and the resulting sanitized delivery facts. Re-deliver duplicate callbacks/recovery triggers only through safe provider/test controls and prove they do not create a second email.
  4. Do not deliberately manufacture a provider ambiguity. If one occurs naturally, preserve outcome_unknown, do not retry, and use the permanent admin reconciliation only after independently establishing the real-world effect.
  5. Automated acceptance never sends to a real sponsor.
Stage D — rollback and closeout

Disable the kill switch through ordinary reviewed configuration and prove new preview/approval/dispatch is blocked while history and existing immutable provider facts remain readable. Re-enable only if the rollout decision explicitly requires it and all Stage B conditions still hold.

A destructive production restore/table replacement is not a release exercise for this issue. Any real restore remains separately reviewed and begins with sending disabled and restored attempts recovery-blocked.

Acceptance criteria

  • Permanent reviewed-communications source is independently tested and PM accepted.
  • Sponsor Finance and Communications compose with separate routes, flags, errors, cron isolation, and rollback; #113 is not a functional prerequisite for #114.
  • Accepted source is on main and was deployed successfully through normal OIDC CI with sending disabled and production smoke green.
  • The final Sponsor table is CloudFormation-owned, empty, and has the four required active indexes.
  • Migration-only suppression/orphan APIs, framework code, UI, and ongoing tests were deleted under #174; live uncertain-send and suppression safety remain.
  • [HUMAN] Stage A private ownership/template/sender/suppression/owned-inbox decision is recorded without public private data.
  • [HUMAN] After #166 cleanup, Stage B proves exact private configuration, effective IAM/event sanitization, default-off behavior, and ordinary-CI enablement.
  • [HUMAN] Stage C sends exactly one reviewed message to the owned inbox and proves truthful accepted/delivery state and no duplicate effect.
  • [HUMAN] Stage D proves disable/rollback behavior and records the final rollout decision.

Close only after all four HUMAN criteria have accepted sanitized evidence. A source push, secret creation, flag change, accepted provider response, or one screenshot alone is insufficient.

Dependencies and composition

  • #111 is complete and supplies authenticated Sponsor organizations, contacts, bookings, history, and alerts.
  • #113 is an independently default-off finance product. Its source composes with #114, but invoice/payment semantics, finance data, and finance enablement do not gate a reviewed communication.
  • #129 and the #136/#143/#147 table/permissions chain are complete. Their accepted source and default-off deployment evidence are historical satisfied dependencies, not current blockers.
  • #166 is the only application-publication blocker: no #114 configuration-changing push or real send until its final schema, canonical first-write, cleanup, and ordinary-deploy restoration are accepted.
  • #174 is complete. No deleted migration machinery may be recreated to make the HUMAN release easier.
  • Private templates, HMAC material, provider credentials/configuration, owned inbox, and a canonical Booking/contact are HUMAN/data readiness gates, not agent implementation work.

Out of scope

  • Importing historical Sponsor data or running any retained importer.
  • Reintroducing migration, orphan, rollback, checkpoint, backfill, or compatibility APIs.
  • Finance/invoice/payment behavior from #113.
  • Generic campaign/workflow/plugin systems, automated drafting/approval, bulk mail, HTML, attachments, multiple recipients, inbound replies, or sending to a real sponsor during acceptance.
  • Publishing private templates, addresses, contacts, provider output, secrets, operational links, or restore artifacts.
  • Manual app deployment or any modification to the protected #166 phase chain.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No new source implementation is open: the accepted TypeScript source is deployed default-off, with integration TEST PASS evidence and CI run 31711997388 linked in the issue. Start with Stage A's private decision, then wait for #166 cleanup before authorized operators follow Stages B-D. Done means sanitized acceptance of all four HUMAN criteria, including one owned-inbox send and rollback; backend/scripts/import-sponsor-crm.ts is explicitly out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, github-actions, typescript
Domain
backend, cloud, database, devops, frontend, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.