DataTalksClub / DataTalksClub/website

Migrate edition-like Course records to reusable Course → Cohort

Open
#51 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

courses data-migration enhancement foundation P0 testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Parent epic: #5

Outcome

Finalize and lifecycle-accept the repository Course → Cohort structural migration so downstream authorization, lifecycle, profile-registration, and full-rehearsal work can depend on one exact identity and relationship contract.

The current repository already contains Course/Cohort source and migrations 0042–0051. That is source-only adoption/proposal history: it does not prove #51 acceptance, a reviewed legacy mapping, a supported populated-data transition, production activation, or a production migration. The #51 engineer starts from that current state after #224 lands, audits it against this contract, and repairs only the remaining structural/migration gaps. Do not replay the stale 3093607 branch or replace proven course behavior.

Normative authority

Hard dependency and dispatch gate

#30, #14, and #15 are accepted decisions/baselines. The remaining hard input is #224: its reviewed edition→family mapping, removal of live regex inference, historical certificate coverage, and legacy-identifier schema must be independently accepted, integrated, green on current main, and identified by exact merge SHA before #51 engineering starts.

An accepted local #224 candidate or a source commit on an unmerged ref is not an integrated dependency. If #224 intentionally retains a later production-execution remainder, PM must first split or move that remainder to #60 and close the repository slice; an open hard dependency must not be bypassed.

Engineering then starts in one fresh isolated worktree from the exact green current-main base, records the final accounts and courses migration leaves and #224 mapping digest, and freezes a new verification plan. Any base, mapping, migration-graph, or source-provenance drift returns #51 to PM.

Exact accepted interface

Stable Course and Cohort identity
  • Course is the reusable family. Its primary key is an immutable UUID; its canonical family slug is globally unique. It contains family/public metadata only and has no learner, schedule, submission, score, certificate, or finished state.
  • Cohort is the evolved legacy edition row. It keeps the existing courses_course table and legacy integer primary key unchanged, and also has one immutable unique UUID.
  • A migrated Cohort UUID remains deterministic under namespace f07c0f6c-2cb4-4bb3-a44b-8c8f9d2bf4d2 with name legacy-cohort:<decimal legacy primary key>. Replay cannot change it.
  • Each Cohort has one required course parent, one globally unique retained legacy slug, one nonblank slug-like identifier unique within its Course, and year as schedule/display metadata only. year is not identity.
  • Each reviewed family receives one stable UUID recorded by the migration/mapping artifact. A family represented by several legacy editions resolves to one Course row, never one generated family per edition. The UUID assignment and family-owned metadata selection are deterministic and replayable; conflicting nonblank family candidates fail preflight unless the reviewed artifact names the exact winner. No runtime year stripping or fuzzy inference is allowed.
  • Legacy numeric Cohort lookups and the global legacy cohort slug remain compatibility identities. Existing homework/project calendar UID inputs and the permanent @courses.datatalks.club namespace remain byte-stable because their underlying legacy rows and keys are preserved. #51 introduces no new public redirect or route policy.
Mechanically preserved ownership

The existing legacy edition row becomes Cohort without recreating its table or renumbering it. Every edition-owned relation remains attached to that same Cohort row, including registration campaign pointers and registration snapshots, enrollment, homework/question/submission/answer, project/criterion/submission/review/vote/evaluation, leaderboard/complaint, score/statistics, certificate, Wrapped, curriculum, content-type, and permission state.

Existing database column names may remain as compatibility names when renaming them would add risk, but Django relation targets and application semantics must be unambiguously Cohort-owned. One learner may have independent rows in multiple Cohorts of one Course; no submission, review, score, statistic, certificate, registration snapshot, or curriculum row may cross Cohort scope.

Preflight, apply, replay, and rollback

Before any corrective constraint or family consolidation mutates rows, a deterministic read-only preflight reports safe aggregate categories for:

  • missing/unmapped/duplicate edition mappings and family metadata conflicts;
  • duplicate or inconsistent enrollment/student references;
  • homework answers/submissions, project submissions/review pairs, criteria/criterion responses, and same-family identifier collisions; and
  • orphaned relations, legacy-key collisions, content-type/permission drift, and unexpected migration provenance.

Unknown, ambiguous, duplicate, stale, or conflicting state blocks apply without partial mutation. No value is guessed, silently repaired, truncated, or printed with learner/profile content.

Repository verification uses synthetic, production-shaped populated fixtures and proves forward apply, exact replay/apply-twice, injected-failure rollback, supported reverse/compatible-application rollback, and forward reapplication. It compares row counts, stable keys, safe checksums, relation targets, representative derived scores/statistics, certificates, content types, permissions, legacy lookups, and calendar UIDs. Intentional non-reversibility must be named and cannot erase legacy data or prevent the recorded compatible application from reading the retained table.

Source, activation, and production boundary

#51 delivers and accepts repository code, migrations, deterministic mapping consumption, synthetic populated-data evidence, and the stable interface above. It performs no protected-data inspection, production import, deployed migration, writer or route activation, compatibility-field removal, or destructive contraction.

#60 later owns the separately authorized production-like snapshot rehearsal, full reconciliation, backup/restore, freeze/delta, and compatible application rollback across all accepted course workflows. Deployment applies exact-image migrations through the release process; #287 owns later observation/contraction policy. Neither already-present source nor a green local synthetic run is production activation evidence.

Downstream contract

  • #52 consumes only the final Course UUID, Cohort UUID, required parent, retained legacy identities, and cohort-scoped relation contract for authorization assignments and object resolution.
  • #53 consumes the same accepted structure plus the migration/rollback seam for lifecycle, canonical public routes, and duplication. #224 is transitive through #51.
  • #247 consumes historical migration-model access to the final Cohort/CourseRegistration ownership shape for its most-recent-registration profile suggestion. #231 and #234 remain independent direct inputs to #247, not dependencies of #51.
  • #60 consumes #51 only as the accepted structural repository input; #60 is downstream and never a prerequisite of #51.

Acceptance criteria

  • The exact start SHA, #224 merge SHA/mapping digest, migration leaves, source pin, and pre-change #30 characterization result are recorded.
  • Course/Cohort identities and invariants match the exact interface above; multiple reviewed editions of one family converge on one Course, and no generated one-family-per-edition state remains active.
  • Every legacy edition maps exactly once; unresolved/duplicate mapping, family metadata conflict, identifier collision, or malformed legacy identity fails before mutation.
  • The legacy courses_course rows and numeric keys are not recreated or renumbered; every listed operational/history relation remains mechanically cohort-scoped with no cross-cohort leakage.
  • Deterministic preflight covers every named conflict class and emits aggregate/redacted evidence only.
  • Forward, replay, injected-failure rollback, supported reverse/compatible-application rollback, and forward reapplication preserve the required counts, keys, checksums, relations, scores, certificates, permissions/content types, lookups, and calendar UIDs.
  • No production/provider access, live migration, route/writer activation, compatibility contraction, or behavior redesign is present.
  • Focused migration/model/relationship tests, migration drift/history, copied course characterization, security/PII canaries, container checks, and the graph-selected versioned verification plan pass on SQLite and the repository's bounded PostgreSQL migration boundary.
  • A separate tester validates the frozen candidate and PM accepts it before a focused Closes #51 commit is locally merged and pushed.

Required scenarios

  1. Zero, one, and several legacy editions; two and three editions map to one family; unrelated families remain separate; descriptive Cohort identifiers and same-family collisions.
  2. Every reviewed #224 edition plus unknown, duplicate, stale, and conflicting mapping inputs; identical and conflicting family-owned metadata.
  3. A learner enrolled in two Cohorts of one Course with independent homework, projects, submissions, peer reviews, criteria responses, scores/statistics, certificates, registration snapshots, and Wrapped data.
  4. Missing/orphaned learner and relation references; duplicate answers/reviews/criteria; injected failure before and during mutation; concurrent/repeated invocation.
  5. Fresh install, legacy leaf → final leaf, final leaf replay, supported reverse/compatible old application, and forward reapplication with exact migration-provenance assertions.
  6. Legacy numeric and slug lookup plus representative HTML/API/calendar compatibility; stable homework/project UIDs and permanent namespace.

Browser and render evidence

The structural migration introduces no new design. The independent tester runs the graph-selected course browser tier against the frozen candidate and compares representative family/cohort, dashboard, calendar, homework, project/review, leaderboard, and certificate routes. If any rendered input changes, capture and inspect the graph-selected desktop/mobile states under .tmp/screenshots/; otherwise screenshots may be not_applicable only when the computed graph proves no render impact.

Explicit non-goals

No #224 reimplementation; reusable/versioned shared curriculum; lifecycle/public-route/duplication work owned by #53; staff/capability/token/view-as work owned by #52; registration/profile/enrollment redesign; homework/project/scoring/leaderboard/certificate behavior redesign; Studio/admin API parity; production/private-data access; provider/network action; source-pin advance; deployment, redirect/DNS activation, email sending, destructive contraction, commit, push, or merge during engineering/testing/PM review.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

After #224 is integrated, start from migrations 0042–0051 and read specs 04, 09, 10, and _docs/PROCESS.md against the accepted Course/Cohort contract. Record the exact base and mapping details, then verify deterministic preflight, populated synthetic migration scenarios, replay and rollback behavior, relationship preservation, and the listed SQLite/PostgreSQL checks without production activation.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, postgresql, python, sqlite
Domain
backend, database, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.