Make the squashed course migration history safe for populated databases

Outcome

Repair the Django courses migration graph so a release built from current main can safely migrate both a fresh database and a populated database whose migration history comes from the deployed legacy course release. Preserve course data and migration provenance without manually rewriting django_migrations, dropping/recreating tables, or importing a database dump as a workaround.

This is a narrowly bounded migration-history compatibility repair. It is not the Course → Cohort product migration, the full course-platform preservation/cutover rehearsal, or the content release digest repair in #219.

Normative authority and incident evidence

• Process and issue lifecycle
• 04 — Course-platform adoption and Course → Cohort model, especially the expand-and-contract migration and high-risk migration checks
• 09 — Migration and rollout controls
• 10 — Verification strategy
• Related course boundaries: #51 and #60

The exact incident inputs are frozen for this issue:

The failed deployment must not be “fixed” by retrying blindly, deleting migration rows, flushing the database, or treating a migration contract failure as an infrastructure success. Future evidence must identify whether a failure is a migration-graph/data-state defect or a runtime/infrastructure defect using the exact source SHA, immutable image identity, migration-task exit/error, applied migration-state snapshot, and controller/release result.

Frozen compatibility boundary

The implementation must support this exact boundary and no wider one:

1. Fresh: an empty database at migration zero reaches every current course leaf from the repaired graph.
2. Legacy populated: a synthetic, non-production database is created with the course migrations from 11b2bd1 applied through courses.0041_courseregistrationcountsourcerun_and_more, then populated with representative course rows and relationships. The repaired current graph must upgrade it without an inconsistent-history error, data loss, or silent schema replacement.
3. Repaired replay: a database already upgraded by the candidate can run the same migration command again and converge to an unchanged leaf with no duplicate backfill, duplicate rows, new side effects, or checksum/count drift.

The lower boundary is the exact deployed legacy migration identity above. The upper boundary is the candidate’s repaired current course leaf. Databases older than that deployed graph, future Course → Cohort migrations, and arbitrary hand-edited django_migrations tables are outside this issue.

The engineer may use a Django-compatible replacement/bridge/staged migration design after documenting its graph and state semantics. The implementation must preserve the meaning of already-applied legacy migration identities and must prove both fresh and populated behavior. It must not assume that a fresh 0001_initial name means the existing database has the new 0001 schema.

Scope

• Repair only the courses migration graph introduced by 4825aa3, including the compatibility boundary between the deployed legacy chain and the current squashed graph.
• Retain or bridge deployed migration history in a way Django can resolve for existing django_migrations state; do not silently discard the legacy chain.
• Add deterministic synthetic legacy-history fixtures/tests that exercise schema state, applied migration records, representative data, foreign keys, content types/permissions where applicable, and current leaf state.
• Prove safe migration behavior for fresh, populated-legacy, replayed-current, and failure/retry paths. Any data transformation must be an ordinary, historical-apps-only migration operation with a clear reversible/no-op policy; no raw runtime import or external side effect is permitted.
• Update the course migration adoption note only if needed to record the corrected compatibility contract and remaining HUMAN gate. Do not rewrite authoritative specs in this issue.
• Leave the current course application behavior, URLs, APIs, Studio, templates, email/Datamailer behavior, and other app migration histories unchanged.

Exact changed-path boundary

The frozen engineer candidate may change only these paths:

• courses/migrations/** — course migration graph/compatibility modules only;
• courses/tests/test_migration_history_compatibility.py — a dedicated test module if useful;
• test_support/tests/test_migrations.py — the existing migration harness, if extension is required;
• test_support/migration_seeds/courses-legacy-history-v1.json — synthetic, non-PII fixture data if a committed seed is required;
• _docs/adoption/course-platform/migration-squash-gate.md and _docs/adoption/course-platform/verification.md — factual gate/evidence updates only, if needed.

The engineer handoff must list every actual path and the exact base/head, plan digest, graph digest, evidence paths, and diff identity. No tracked path outside that allow-list is in scope. In particular, this issue must not change courses/models/**, views, templates, APIs, services, accounts/**, content/**, content_sync/**, events/**, Makefile, .github/workflows/**, pyproject.toml, uv.lock, Terraform/AWS configuration, or any database/export/protected data.

Explicit non-goals

• No Course → Cohort domain/model/relationship redesign or broader course-platform implementation from #51.
• No full production-like course data migration, route/API consumer migration, redirect/cutover, write-freeze/delta rehearsal, or release authorization from #60.
• No content release digest migration or content_sync behavior from #219.
• No destructive table recreation, DROP TABLE, TRUNCATE, flush, bulk delete/reinsert, manual django_migrations edits, or database dump restore as a compatibility strategy.
• No production database, AWS console/API, protected export, credential, real learner data, email/provider/Datamailer dispatch, or worker side effect.
• No dependency, CI workflow, Makefile, model, route, UI, API, schema-policy, or unrelated migration cleanup.
• No claim that the failed run 32659885289 was fixed, no deployment success claim, and no issue closure before the named HUMAN gates pass.

Dependencies and blocked work

• Required implementation inputs: the exact trees at 11b2bd1, 4825aa3, and b7c693e; the redacted CI/deployment evidence linked above; and the current migration/test harness. No live database is a dependency for engineering or tester work.
• Related, not blocking: #51 is the broader Course → Cohort migration and remains governed by its open decisions/dependencies; #60 is the broader preservation/cutover rehearsal and remains blocked by its own course consumer, mapping, and workflow gates. #220 does not implement, accept, or unblock either issue.
• Release dependency: no release may deploy the current squashed course graph to a populated environment carrying the legacy chain until the automated scenarios pass and the exact-image HUMAN deployment/on-call gates below pass.
• If the repaired graph requires a product-model change or a data mapping decision owned by #51/#60, stop and re-groom rather than widening this issue.

Acceptance criteria

• Migration graph is explicit and compatible. The candidate documents the repaired graph, dependencies, replacement/bridge semantics, and current leaf. showmigrations/MigrationLoader finds no missing legacy dependency, duplicate leaf, or inconsistent applied-history state. No already-deployed migration identity is silently redefined as a different schema.
• Fresh database. An isolated empty database migrates from zero to every leaf with no error. makemigrations --check --dry-run reports no model drift, and the resulting schema matches the current models and expected course table/constraint contract.
• Populated legacy database. A deterministic fixture built from the exact 11b2bd1 legacy graph applies through courses.0041_courseregistrationcountsourcerun_and_more, inserts representative synthetic Course/enrollment/homework/project/review/score/statistics/certificate/registration data and relationships, then upgrades with the candidate. The run succeeds without InconsistentMigrationHistory, missing-table/column errors, guessed mappings, or destructive reset.
• Data and provenance preservation. Before/after evidence compares table row counts, primary keys, foreign keys, representative deterministic row checksums, content types/permissions where touched, and the complete applied migration set. No existing course row is silently dropped, duplicated, renumbered, or recreated under a new identity.
• Replay and retry are safe. Re-running the candidate migration on the upgraded populated database is a successful no-op: migrate --plan is empty at the leaf, row/checksum/migration-state snapshots are unchanged, no duplicate backfill occurs, and no email, provider, Datamailer, queue, or other external side effect is emitted. A failed/retried local migration either resumes deterministically or fails closed without partial destructive cleanup.
• Historical migration isolation. Every data migration uses historical apps models, is backend-portable, has bounded transaction/atomic behavior, and passes the repository’s migration import/side-effect isolation checks. No migration imports mutable runtime services, reads secrets, reaches the network, or depends on current application code that can drift later.
• Repository and regression verification. The engineer generates the versioned selective-verification plan from the frozen candidate, records exact identities/digests and all component dispositions, and leaves an uncommitted frozen worktree. The independent tester reruns the required focused migration/Django checks and make test-playwright-core; any render-impact classification triggers the required desktop/mobile screenshot gate, while a justified no-render classification is recorded as not_applicable per the process. No required component is partial or undocumented.
• Browser/HTTP compatibility smoke. On the synthetic migrated state, representative public course list/detail and private course workflow/API smoke requests retain expected status, canonical/compatibility behavior, no migration traceback, and no accidental data exposure. Desktop/mobile browser evidence is required when the selective plan marks those routes render-impacting.
• [HUMAN] Exact-image populated deployment. After commit, local merge, push, and a successful CI build, on-call deploys the exact immutable image/source identity to the authorized development environment and runs the migration task against an authorized populated development database copy initialized at the legacy boundary. Redacted evidence must include source SHA, image digest/version, migration task result, applied-history summary, release-controller result, and whether the prior release remained protected on failure. No production/AWS mutation is authorized by this issue’s engineering scope.
• [HUMAN] Readiness, smoke, and failure classification. On-call verifies database-aware readiness, web/worker identity parity, representative course HTTP/browser smoke, migration-task logs, and the release artifact. If the task fails, the evidence must distinguish a graph/data-state defect from a runtime/infrastructure defect; a controller rejection or missing verdict is unresolved, not green. The prior release remains the rollback target and no destructive database rollback is attempted.
• [HUMAN] CI/on-call lifecycle gate. The post-push CI run is observed by on-call to a terminal green result with no cancellation, missing required artifact, deployment contract contradiction, or unreviewed exception. The issue remains OPEN with human until these named gates are reported. If a technical commit is later authorized while the HUMAN gates remain, its body must use Refs #220, not Closes #220.

Required verification commands

Use the repository’s uv-backed commands only; all local migration databases and fixtures must be isolated under project-local .tmp/ or the deterministic test harness.

uv sync --locked
uv lock --check
make migrations-check
make test-migrations
make verification-quality
make test-django-full
make test-playwright-core
make verification-container
make verification-plan VERIFY_BASE_SHA=<frozen-base> VERIFY_HEAD_SHA=<frozen-head> VERIFY_ISSUE=220 VERIFY_WORKTREE=<frozen-worktree> VERIFY_CONSUMER=engineer VERIFY_PHASE=engineer
make verification-run VERIFY_ISSUE=220 VERIFY_WORKTREE=<frozen-worktree> VERIFY_CONSUMER=tester VERIFY_PHASE=tester
make verification-evidence-check VERIFY_ISSUE=220
make verification-report-check VERIFY_ISSUE=220 VERIFY_PHASE=tester

The handoff must include the exact commands actually run, terminal exit statuses, test counts, migration plan/output, redacted before/after snapshots, screenshot disposition/paths when applicable, and validated plan/evidence/report paths. make test-migrations and make migrations-check are required even if a broader target also invokes them. Do not run these commands against the shared development database.

Handoff and disposition

Engineering starts only after this issue is groomed and needs grooming is removed. The engineer must leave the candidate uncommitted and frozen for an independent tester; the tester must report every automated criterion separately; PM acceptance is a later lifecycle gate and is not granted by this issue rewrite. The current failed deployment evidence is preserved as the baseline incident and is not a success signal.

Compare the course migration trees at 11b2bd1, 4825aa3, and b7c693e, then inspect courses/migrations/** and the migration harness in test_support/tests/test_migrations.py. Run showmigrations, MigrationLoader checks, and the existing migration tests before adding the dedicated compatibility test and any listed fixture. Done means fresh, populated legacy, replay, and retry scenarios preserve data and migration provenance without destructive resets or side effects.