DataTalksClub / DataTalksClub/dataops

Prepare and perform the retained one-off Trello Card import for V1

Open
#41 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend data enhancement human P1 work-engine
Dominant language
TypeScript
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Prepare and perform the retained one-off Trello Card import for V1

Status: blocked — HUMAN inclusion is recorded and the technical canonical baseline is satisfied; Gate 2 waits only for the explicit HUMAN data-model stability declaration below
Tags: enhancement, work-engine, backend, data, human, P1
Depends on: immediate Gate 2 dependency is the remaining HUMAN Gate 1 declaration; Gates 3–6 remain required for private source review and execution; #217 then #146 must finish before Gate 5 target/recoverability/write authorization
Blocks: #71 because the HUMAN owner included the historical Trello Card import in V1
Next owner: HUMAN data-model owner for the one explicit stability declaration; after that, Software Engineer for the bounded public-code Gate 2 slice
Resume condition: a HUMAN data-model owner declares the listed import-relevant contracts stable for this one load; this declaration authorizes only PM/SWE/Tester Gate 2 work and does not authorize source access, importer execution, AWS access, backup/export/restore, or target writes

Product decision

Keep backend/scripts/import-trello-csv.ts as a manual one-off raw-source tool. The HUMAN product/data owner included the historical Trello Card import in V1 on 2026-08-30; that decision authorizes the bounded public-code readiness lifecycle only after Gate 1 is complete.

This is not a migration framework or shipped product capability:

  • no runtime/API route, Lambda handler, schedule, deploy step, seed, normal CI command, or permanent migration state;
  • no reconcile, rollback, orphan-management, dual-read, compatibility, backfill, or rerunnable upsert service;
  • no live Trello API connection and no Trello mutation;
  • no ongoing importer test suite;
  • no import, source read, export, backup, restore, or target-data read before the gates below.

#71 now treats this import as part of V1 acceptance. The inclusion decision does not authorize private source inspection, an importer/dry-run invocation, AWS/provider access, recovery operations, or a write; those remain gated below.

Current evidence and why the old acceptance is historical

The June #41 implementation and fixture evidence targeted the retired work-engine/ package and pre-canonical Bundle model. Consolidation later moved the retained raw tool to backend/scripts/import-trello-csv.ts, renamed Bundle vocabulary to Card, removed migration-only test suites, and excluded importer inputs from packaged Lambda artifacts.

Accepted #168 commit 41dca9e3748fe627c195f48a7f97ae203b343f60 updated the retained script to compile against the canonical Card/Task repositories, and accepted #179 commit 02ee455efad09865ecdbd578e940c59e0bf7dc36 established canonical Task concurrency/history. Both were deployed during the completed #166/#182 activation. The current rewritten origin/main line preserves their exact patches as 66c72de2c23b5ed192e03d9b4d7b3d345ffb93c2 (#168) and 51285fa2a02da3710c98e71f033257a03a0c2736 (#179); both are ancestors of deployed SHA 6236865e509c0e142d364e6c56f7856d8f932076. Current deploy run 33015955502 passed.

Accepted #187 commit 50df377d04585d625b219fcfb249601847e2976c is likewise preserved by patch-equivalent current-line commit 4c958c1d700ce472687a6d814ecd4a4d44b662fa, an ancestor of the same deployed SHA. Normal CI keeps test:one-off:dry-run-import outside its selected test path.

#217's agent-verifiable identity-stability change is accepted and committed as 676a4f66cb4aafa2eef13287bc9c06e6c5f6e115, but its HUMAN no-execute preview is still pending. #146 therefore remains unshipped. Neither issue blocks Gate 2 because Gate 2 is public code review with no source, client, provider, or target access; both must be resolved before Gate 5 can identify and authorize the sandbox target and write conditions.

The retained Trello script itself still needs a bounded readiness change before any real use:

  • omitting a mode currently selects all Trello/CSV/template paths;
  • omitting --dry-run currently selects write behavior;
  • its header still describes the retired local dynalite/LevelDB shape;
  • it preflights and writes one Card plan at a time, then creates Tasks/artifacts/audit records sequentially, so a later collision or failure can leave a partial batch;
  • its historical “migration” provenance and automatic artifact/follow-up behavior have not been accepted against the final #168 operator semantics.

Existence and compilation therefore do not constitute import readiness.

Exact scope if #71 includes the import

Import only active, non-template, non-closed Trello Cards from the HUMAN-approved source lists that map to:

Approved Trello list Canonical Card stage
Preparation preparation
Announced announced
After event after-event

Use only the explicit Trello Card mode. Trello templates, closed/Done Cards, CSV tasks, completed CSV history, and the script's default “all” mode are outside #41. The CSV reader remains retained for its separately owned future decision; this issue neither runs nor deletes it.

The source is a human-provided local Trello JSON export. The operator never fetches it from the live Trello API. Raw export contents, paths, Card/checklist text, member identifiers, attachment URLs, and private report output stay out of this public repository and issue.

Gate 1 — canonical data-model freeze

All conditions must be recorded before source inspection or a dry-run:

  • #166 repaired A/B/C, #182's final no-replacement Cards preflight, reviewed D, and cleanup passed in the sandbox; temporary phase controls were removed and ordinary push OIDC deployment is green.
  • Canonical #179/#168 behavior was activated and remains on the current deployed line. The accepted patches are preserved by current-line commits 51285fa2c and 66c72de2c; canonical create/read/mutate/archive/reopen evidence passed with no legacy rows, fallback, backfill, or compatibility shape.
  • #187's accepted patch is preserved by current-line commit 4c958c1d7; normal CI proves that one-off dry-run-import verification is not selected.
  • HUMAN data-model owner declares the import-relevant Card, Task, artifact/link, provenance, waiting/follow-up, proof, assignee, and process-doc contracts stable for this one load.
  • Gate 2's output contract is the #168 shape: every planned Card has version >= 1, system-owned taskCount/openTaskCount, and exactly one active or archived stage/status pair; every Task has version >= 1, taskHistory, an exact status, and canonical Card membership. Actual source plans are reviewed only in Gates 3–4.
  • Canonical aggregate semantics win over Trello list labels. Gate 2 must deterministically report zero-Task and all-terminal outcomes; the HUMAN source owner excludes, remaps, or accepts actual cases in Gates 3–4 before any write.
  • The product transaction boundary is fixed for Gate 2: a Card must not become publicly visible with partial Tasks/counters or misleading completion/reactivation history. Any Card that cannot fit the reviewed atomic boundary is excluded and reported; it is never split into a partial visible result.

If the model is still changing, keep #41 blocked. Do not add a compatibility reader or teach the importer two schemas.

Gate 2 — bounded script readiness, without data movement

Only after the remaining Gate 1 HUMAN declaration, a Software Engineer may make the smallest script-only readiness change. It must pass the normal lifecycle before any private source is read.

Permitted implementation surface is the retained importer plus the minimum static/compile-time verification needed to prove these criteria. It may reuse existing canonical repository primitives but may not change API, frontend, Lambda/runtime behavior, infrastructure, workflows, packaging, seeds, schedules, or normal CI selection. If atomic Card/Task creation cannot be achieved without changing shipped runtime behavior, stop and return to PM for a separately groomed scope; do not broaden Gate 2 in place:

  • Require exactly one explicit mode and an explicit --source; remove the default “import all/bundled source” behavior.
  • Dry-run is the safe default. A write requires a separate explicit write flag plus a target-confirmation value; absence or mismatch fails before client creation.
  • The command reports source checksum, code SHA, planned deterministic IDs/counts, Card/Task final aggregate, warnings, and exclusions without echoing private content into public logs.
  • Preflight the complete deterministic target-ID set before the first write. Any existing Card, Task, artifact, duplicate source ID, or changed plan stops with zero writes.
  • Commit each accepted Card plus its canonical Tasks/counters/lifecycle atomically through the #168 contract, or exclude it before the run. Optional artifacts/audit/notifications must not turn a failed side write into a partially accepted Card.
  • Replace migration-framework terminology with one-off import terminology. Do not add a migration table, run record, API, rollback/reconcile endpoint, compatibility path, or scheduled cleanup.
  • Preserve the runtime packaging boundary: the importer remains absent from every SAM artifact, handler graph, workflow, seed, schedule, and normal CI path.

Verification for this readiness slice is static scope review, scripts TypeScript compilation, the existing runtime packaging boundary, and diff inspection. Do not run the importer, a raw export, a restore, the broad backend test glob, or test:one-off:dry-run-import. Do not create a new ongoing fixture/import suite. The manual #187 command may be invoked only later as part of an explicitly authorized one-off operator event.

Gate 3 — HUMAN source and privacy approval

The HUMAN source custodian records privately, with only sanitized disposition posted here:

  • exact board/export owner, export timestamp, file checksum, approved lists, and selection cutoff;
  • whether descriptions, comments, labels, member/assignee hints, due dates, checklist completion, attachments, links, and source IDs may be imported;
  • a redacted source copy or exclusion list; no token, cookie, signed/private URL, contact detail, sponsor/finance context, or unrelated operational content survives;
  • allowed attachment/link domains and data classifications; “syntactically safe URL” is not sufficient privacy approval;
  • exact assignee/user mapping and treatment of unknown members;
  • explicit handling for unresolved process docs, workflow type, proof inference, waiting/follow-up inference, invalid date, zero-Task Card, all-terminal Card, duplicate source ID, and oversized Card;
  • private retention/deletion owner for the raw export and private dry-run report.

No raw source, checksum-to-path mapping, report body, private URL, or operator identity is posted publicly.

Gate 4 — one authorized dry-run

Use the exact reviewed code SHA and approved source checksum. Run only the explicit Card-only dry-run; never the default/all/templates/CSV modes.

The HUMAN reviewer must approve:

  • source checksum and code SHA exactly match Gate 3 and the reviewed readiness artifact;
  • counts for source Cards, selected/excluded Cards, Tasks, terminal/open Tasks, zero/all-terminal Cards, artifacts/links, proof fields, waiting/follow-ups, assignees, and process-doc mappings;
  • every warning/error/exclusion is resolved or explicitly accepted; duplicate IDs, invalid canonical dates/stages/statuses, private/unsafe URLs, unknown target users, counter mismatches, and oversized/partial Card plans are hard stops;
  • exact final Card lifecycle/counter results match #168, including any Card that would archive immediately;
  • approved maximum item count, transaction count, runtime, and cost/load boundary;
  • the dry-run performs no DynamoDB client creation or write and no provider/network action.

Post only sanitized aggregate counts, source/code digests, warning categories, and PASS/FAIL.

Gate 5 — HUMAN recoverability and write authorization

Immediately before a write, while the exact source, code, plan, and target remain unchanged:

  • identify the exact sandbox account/region/stack and stack-owned Cards, Tasks, artifact/audit/notification tables privately; production is out of scope;
  • verify no #166 phase/deploy/update is active and establish a bounded Card/Task/import/notification maintenance window;
  • create and verify fresh recoverability evidence for every table the plan can touch plus a fresh portable product export/validation; record private locations/checksums and sanitized statuses only;
  • re-prove every planned deterministic target ID is absent and the live schema/permissions match the reviewed plan;
  • choose the failure disposition before execution: stop and preserve a partial result for a separately groomed recovery, or use a separately authorized restore/reset. Ad hoc rerun, per-record deletion, orphan reconciliation, or improvised rollback is forbidden;
  • HUMAN source owner, data-model owner, sandbox owner, and credentialed operator explicitly approve the exact single write command, SHA, source checksum, target, counts, and time window.

#41 does not itself authorize AWS, backup, export, restore, table replacement, or write actions. Those occur only after the explicit HUMAN gate.

Gate 6 — one-shot write and post-write acceptance

The credentialed operator runs the exact approved command once. On any mismatch, warning growth, collision, timeout, deployment overlap, or partial failure: stop. Do not rerun.

Before closure:

  • created Card/Task/artifact/audit/notification counts exactly match the approved plan, with zero update/upsert of pre-existing rows;
  • read-only integrity proves canonical versions, Task history, Card counters/lifecycle, membership, proof, waiting/follow-up, provenance, and no forbidden/private fields;
  • the operations manager samples every stage and all exception categories through normal Cards/Tasks/Archive surfaces and confirms imported work is understandable and actionable;
  • a fresh post-import portable export validates and records only private archive details plus public-safe counts/checksums/status;
  • notification/provider effects remain off unless separately authorized; importing rows is not authorization to message anyone;
  • sanitized outcome and any deferred/excluded records are cross-posted to #71.

Then PM may close #41. The retained raw tool remains manual and outside runtime/CI; whether it is later deleted or retained for another approved source load is a separate explicit decision.

Acceptance criteria

  • The raw Trello/CSV importer is retained as source-only tooling and excluded from packaged runtime/deploy/API/schedule paths.
  • #71 explicitly records included or deferred. Deferred performs no data action and does not block the core sandbox.
  • Gates 1 and 2 establish one canonical, fail-closed, manual-only importer with no migration framework or ongoing test execution.
  • [HUMAN] Gates 3 and 4 approve the private source/redaction/mapping and exact no-write dry-run.
  • [HUMAN] Gate 5 approves exact sandbox recoverability, target, plan, failure disposition, and one write command.
  • [HUMAN] If included, Gate 6 completes one write and sanitized post-write acceptance; if deferred, no import evidence is fabricated.
  • No live Trello call, production target, default/all/template/CSV mode, compatibility/upsert, importer rerun, migration API, or normal-CI importer test is introduced.

Out of scope

  • Reading or changing live Trello.
  • Trello Template import, archived/Done history, or CSV task import; retain those raw readers for separately owned future decisions.
  • Redesigning the canonical #168 model to accommodate historical source rows.
  • Production deployment/data, bidirectional sync, recurring sync, or a supported import product.
  • Permanent migration records, reconciliation, orphan management, rollback routes, compatibility, or backfill.
  • Running one-off import/export/restore/migration tools during grooming, implementation review, normal CI, deploy, or unrelated verification.
  • Publishing raw operational data, private paths/URLs, credentials-adjacent information, or archive locations.

Evidence boundary

This issue may publicly record issue/run links, exact public code SHAs, source checksum only if it cannot reveal a private location, aggregate counts, warning categories, PASS/FAIL, and sanitized timestamps. Keep raw source, item text, target identifiers, private archive details, provider configuration, and operator identities private.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

First verify the pending HUMAN data-model stability declaration before touching the code. Then inspect backend/scripts/import-trello-csv.ts and perform the permitted static scope review and TypeScript compilation; the Gate 2 readiness criteria define done, including explicit Card mode/source, safe dry-run behavior, deterministic preflight, atomic canonical writes, and preserved packaging boundaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.