DataTalksClub / DataTalksClub/dataops

Epic: Deliver a Telegram-first conversational agent MVP

Open
#122 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

assistant backend data enhancement human infra P1 testing
Dominant language
TypeScript
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Epic: Deliver a Telegram-first conversational agent MVP

Status: blocked overall — all seven child source slices, reviews, deployment permissions, and the normal-CI all-off deployment are complete; no direct epic implementation remains, and the consolidated HUMAN rollout waits for #166/#182 to restore the final canonical Card/Task baseline
Tags: enhancement, assistant, backend, infra, testing, data, P1, human
Depends on: #166 A/B/C → #182 final no-replacement Cards preflight → #166 D → cleanup, followed by one successful ordinary all-off deployment; then #128's staged HUMAN rollout and child evidence cross-posts
Satisfied dependencies: #123 is closed; #121 and #124-#128 source/default-off prerequisites are integrated; #129/#130/#131 and #136/#140/#143 are complete; normal dark deployment is proven
Not a dependency: #137 retained-queue evidence; any migration/import framework, polling bot, source repository, or manual app deployment
Blocks: none outside this epic; #128 closes #121 and #124-#127 from its cross-posted HUMAN evidence, then closes #122 last
Next owner: #166 credentialed operator and On-Call for the protected storage sequence; afterward the authorized #128 Telegram/provider owner with On-Call; PM closes each child and this epic
Resume condition: #166/#182 record accepted production outcomes, canonical writers reopen, temporary phase controls are removed, ordinary push-triggered OIDC deployment is restored, and one following normal main run succeeds with the exact six conversational controls dark
Architecture: docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md (accepted architecture commit 4e5402c)
Current evidence: accepted child commits listed below; normal dark deployment 31545689094; reconfirmation 31711997388; current #128 rollout tracker

Epic outcome

Ship one private-Telegram conversational MVP for verified DataOps users:

  • text, one bounded voice note, and one bounded photo enter through the same authenticated private-chat adapter;
  • provider-derived transcript/description is private and untrusted until the user explicitly uses, corrects, or discards it;
  • the first internal effect is exactly one actor-owned todo after an immutable preview and transactional approval;
  • the first external effect is exactly one public-source Typefully saved draft after an immutable preview and transactional approval;
  • the Typefully result remains unscheduled, unpublished, and unshared;
  • duplicate delivery, stale controls, wrong actors, revoked bindings, groups, and ambiguous provider effects cannot create a second or unauthorized effect;
  • rollback returns to authenticated bounded dark maintenance and never restores legacy /todo, /social, /podcast, direct mutation, polling, or a second adapter.

The accepted child issues remain the detailed product authorities. #122 coordinates lifecycle and closure only; it owns no additional route, schema, worker, provider adapter, UI, infrastructure, migration, or compatibility implementation.

Completed source and dark-deployment ledger

Milestone Issue Accepted source Current lifecycle
M1 state/storage/retention/export safety #123 e27f6856fee5cc98b94e66cd3a4941195568bd7a closed, deployed, On-Call accepted
M2 static plugin runtime and conversational z.ai #125 543d63c51795b5a8849fc4ba0d8b7802639c51a3 source/default-off deployed; live z.ai evidence remains
M3 Telegram identity/text/voice/photo #121 253d0f9699dae40c3774955514b2e9fe1ba2ef28 source/default-off deployed; HUMAN stages tracked and delegated to #128
M4 transactional approval/execution recovery #124 840dd471a522b9275a62174185076eb6d1072080 source/resources default-off deployed; live worker/recovery evidence remains
M5 conversational todo #126 c47c5258a08b29c7c5ea6e797149c8a6619bbe71 source/default-off deployed; one controlled todo journey remains
M6 Typefully saved draft #127 92706b37abb47616fd071606193066a7b3b735a3 source/default-off deployed; private config/effect/reconciliation evidence remains
M7 integrated rollout/cutover #128 9595361a0546a52a5cb90ace7df0bbdeb524866a plus accepted E2E repair fe030ad3b36e73df2542ca8c2bf42afa7ed3277e all-off deployment complete; staged HUMAN matrix waits for #166/#182

Every commit above is an ancestor of deployed SHA 794076354048f6ff3417d55d13d334caf4237cea.

Normal OIDC run 31545689094 created the accepted dark realization, reached UPDATE_COMPLETE, seeded, and passed deployed smoke. It retained the topic, three custom log groups, and disabled generated schedules; the accepted transform has 17 condition-gated alarm definitions and correctly instantiates zero alarms while all capabilities are off. Run 31711997388 reconfirmed stack deployment, seed, and smoke with the exact dark controls:

Telegram ingress=false
execution=false
plugins=none
Typefully external=false
voice=false
photo=false

This proves publication and default-off infrastructure. It does not prove a real binding, webhook ownership, Telegram delivery, model/media/provider result, task/Typefully effect, queue/heartbeat canary, subscription, or reverse rollback.

Permanent safety boundary

  • Telegram webhook secret/chat allowlisting is ingress control; permanent admin-managed numeric identity binding plus an enabled DataOps user grants private authority.
  • Core, not model/plugin/Telegram, owns revision, request changes, exact approval, cancel, discard, and session transitions.
  • Approval atomically consumes one presentation, claims the exact proposal, and queues one durable attempt. A leased worker performs the effect outside the webhook.
  • Todo is create-one-only and actor-owned. Typefully is public-source create-one-saved-draft-only. No schedule, publish, share, update, bulk effect, or direct slash-command mutation exists.
  • An ambiguous post-dispatch result becomes outcome_unknown and is never automatically retried. Reconciliation cannot fabricate, replay, or overwrite an effect.
  • Raw media reaches only its intended provider through bounded temporary storage and is deleted. Private messages, derived text, proposals, and private results follow the accepted 30-day boundary; minimal safe audit/effect receipts follow the one-year boundary.
  • Public source, logs, issues, screenshots, metrics, queues, failure artifacts, portable exports, and model context exclude credentials, tokens, Telegram identities, raw media, private transcripts/OCR, provider bodies, replayable approvals, and private edit URLs.
  • Production resources remain CloudFormation-owned. Runtime creates no infrastructure. Feature changes deploy only through ordinary reviewed main OIDC CI/CD.

No migration or compatibility work

The MVP needs no data migration/import phase. Identity bindings and conversations are ordinary permanent runtime records created through the accepted product paths.

Do not import usernames/chat allowlists/podcast-bot state, revive a legacy handler, add snapshot/backfill/checkpoint/resume/orphan/rollback APIs, run retained importers, or modify source repositories. #174's migration cleanup does not remove permanent product recovery, idempotency, uncertainty handling, retention, export validation, or disaster-recovery safety.

#166/#182 is application deployment sequencing, not conversational data migration. The exact protected phase commits must not acquire Telegram flags, secret changes, bindings, provider calls, canaries, or epic implementation.

Remaining rollout and closure sequence

#128 is the only execution tracker for the steps below. Child issues do not run parallel provider canaries or independent deployments.

  1. Complete #166 A/B/C, #182's final read-only Cards preflight, #166 D, canonical first-write evidence, and cleanup. Restore ordinary push-triggered OIDC deployment and record one successful all-off run with final writers open.
  2. [HUMAN] Privately confirm one webhook owner/no poller, intended identity/admin, exact managed Telegram/z.ai/Groq/Typefully configuration, alarm subscription owner, public-source/media test fixtures, rollback owner, and public redaction plan.
  3. [HUMAN] Reconfirm dark maintenance/readiness and real bind → revoke → fail-closed → reactivate behavior with zero model/media/domain/provider effect while disabled.
  4. [HUMAN] Enable ingress/execution with plugins todo, media off, and Typefully external off. Prove private text/session behavior, z.ai conversation, exact preview/revision/approval, duplicate convergence, one queued/leased attempt, one actor-owned Task, result delivery, static /todo, and group/unlinked isolation.
  5. [HUMAN] Enable voice and photo separately. Make exactly one bounded Groq voice and one bounded z.ai photo call, cover a use/correction decision across one and discard across the other, prove no retained raw media, and return each media control off after its canary.
  6. [HUMAN] Add Typefully to the allowlist with external execution off. Propose/revise/cancel and prove there is no approve/queue/provider path and no durable dispatchable canary.
  7. [HUMAN] After #127's private account/source/reconciliation readiness, enable Typefully external execution and approve exactly one typed-public-source saved draft. Prove exact intended account/content, unscheduled/unpublished/unshared semantics, one provider effect, private result delivery, and conservative handling of any naturally occurring ambiguity without blind retry. Do not deliberately manufacture an ambiguous provider effect.
  8. [HUMAN] Observe bounded readiness, queue/worker/recovery/result-delivery, heartbeat schedules, confirmed alarm canaries, and redacted logs/metrics. Do not inspect/publish payloads, purge queues, or mutate unrelated retained resources.
  9. [HUMAN] Reverse the controls through ordinary OIDC in the accepted #128 order until the exact all-dark snapshot is restored. Prove durable state remains truthful, no unknown effect becomes retryable, and no legacy mutation/poller returns.
  10. Cross-post sanitized evidence to #121 and #124-#127. PM closes each child only when its own criteria pass, closes #128 after full observation/rollback, then closes #122 last.

A failed stage stops progression and uses the reviewed switch rollback. It does not authorize a manual deploy, Lambda/IAM patch, queue purge, data edit, provider replay, broad credential inspection, import/export/restore, or destructive cleanup.

Epic acceptance criteria

  • All seven child source slices passed their required independent engineering/specialist/Tester/PM gates and were integrated in dependency order.
  • The accepted child source, strict six-control rollout, OIDC permission repairs, and conditional observability graph are on main.
  • Normal OIDC CI successfully deployed the all-off graph, completed runtime seed and smoke, and later reconfirmed the same dark baseline.
  • Permanent identity/privacy, exact approval, durable execution, one-effect idempotency, uncertainty, retention, provider-egress, no-legacy-fallback, and kill-switch contracts are automated and accepted in their child issues.
  • #123 is closed with deployed storage/export/restore safety evidence.
  • #166/#182 complete the final Card/Task production sequence and one following ordinary all-off deployment succeeds before any live conversational canary.
  • [HUMAN via #128 → #121] Real identity/text/session/isolation plus one bounded voice and one bounded photo canary pass with media/privacy cleanup and exact dark rollback evidence.
  • [HUMAN via #128 → #125] The deployed conversational z.ai configuration and one bounded redacted real model interaction pass without secret/provider-body exposure.
  • [HUMAN via #128 → #124] Approval, leasing, Stream worker, recovery/readiness/queue/alarm/result-delivery, uncertainty, and switch rollback pass with sanitized evidence and one effect only.
  • [HUMAN via #128 → #126] One private revise/approve todo journey creates exactly one matching actor-owned Task; duplicate approval and /todo create none.
  • [HUMAN via #128 → #127] Private Typefully config/source authority and one exact saved-draft effect pass; the result is unscheduled/unpublished/unshared and any natural ambiguity is never replayed.
  • [HUMAN] #128 completes the integrated canary/observation/reverse-rollback matrix and returns the deployment to the exact all-dark snapshot.
  • Sanitized cross-posts let PM close #121 and #124-#127, then #128; #122 closes last with all children terminal.

No direct #122 Software Engineer stage remains. If a HUMAN stage finds a source defect, stop and regroom it on the owning child (or a new bounded bug); do not implement an unreviewed fix in the epic.

Out of scope

Telegram groups/shared context beyond static redirect; web conversation/approval; generic files/uploads; SOP/podcast/GitHub mutation; workflows/recurring/calendar/newsletter/sponsor/bookkeeping plugins; scheduling/publishing; history-wide retrieval or automatic memory; dynamic plugins/packages/DSLs; multi-effect plans; plaintext credentials/secret UI; raw operational knowledge in the public repo; manual app deployment; queue/data cleanup; migration/import; and edits to ../dtc-operations, ../datatasks, ../podcast-assistant, or other source repositories.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md and the current #128 rollout tracker; this epic explicitly owns lifecycle coordination only and has no remaining direct implementation. Review the recorded deployment evidence and remaining #166/#182 and HUMAN rollout gates. Done means the protected sequence, ordinary all-off deployment, staged rollout, rollback, evidence cross-posts, and child closures are complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
backend, ci-cd, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.