DataTalksClub / DataTalksClub/website
Repair sync-content verification test after staged-import helper removal
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The maintained Django suite cannot collect because scripts/tests/test_sync_content_verify.py imports both content_sync.tests.helpers and scripts.prod.sync_content_verify, which were intentionally removed with the retired DataTalksClub/content staged-import pipeline in 1cd976a4. scripts/content.py still exposes checkout and drift commands that dispatch to that missing verifier.
This is not a request to resurrect the retired pipeline. It is a baseline repair: remove the orphaned test/CLI surface while retaining every verification invariant that still applies to the supported database-owned sync paths.
Authority
_docs/architecture/database-only-content.md— public content is database-owned; sync may write rows directly, but a checked-in/staged projection is not a runtime authority._docs/runbooks/data-ingest.md— current ingest entry points and source ownership._docs/specs/10-verification-strategy.md— maintained Django gates must collect and pass; a failed gate is not waived._docs/PROCESS.md— selective evidence does not permit a required failed tier to be skipped.
Product/architecture decision
Do not restore content_sync.tests.helpers, content_sync.dtc_content, or scripts/prod/sync_content_verify.py. The removed helper was coupled to the removed dtc_content fixture tree and verifier, not a supported reusable test boundary. Restoring it would make collection green by reviving code for the wrong architecture.
The orphaned verifier test may be removed or replaced only after its still-relevant guarantees are explicitly mapped to current tests and any real gaps are ported to the owning current boundary. Cases that existed solely to compare a retired ContentRelease staged projection with that source checkout are obsolete behavior, not coverage to recreate.
Scope
- Reconcile
scripts/tests/test_sync_content_verify.pywith the removal commit so no maintained test imports or executes deleted staged-import code. - Audit the old test cases and record their disposition in the implementation handoff:
- current equivalent already covered, with the exact test named;
- ported to the current owning test module; or
- obsolete with the retired staged verifier, with the reason.
- Preserve/port the applicable guarantees: source scoping; discover/create/update/delete behavior; revision/checksum provenance; idempotence; fail-closed invalid input/checkout behavior; atomic failure; current offline/no-network paths; and no cross-source deletion.
- Remove the stale
scripts/content.py checkoutanddriftcommand paths and their parser expectations, or retarget them only if a currently supported equivalent already exists. No command advertised byscripts/content.py --helpmay point at a missing module. - Update only directly affected evergreen command/runbook references so operators are not told to invoke removed commands.
- Add a bounded regression that catches dangling production-script/test imports or advertised
scripts/content.pysubcommands before the full suite reaches collection.
Non-goals
- Recreating the deleted staged-import adapter, fixtures, projection comparator, or
ContentRelease-based editorial workflow. - Designing a new generic drift-reporting product or changing the
community_base.content_syncAPI. - Changing public content, parser output, database rows, source registrations, course curriculum ingest, or production data.
- Weakening, skipping, quarantining, or excluding the maintained Django suite.
- Refactoring unrelated legacy importers or resolving all historical runbook drift.
Dependencies and blocked work
- No open implementation dependency. Commit
1cd976a4is historical context, not a dependency to reverse. - This repair blocks final full-Django evidence for #425 and any other candidate currently encountering the same baseline collection failure. Those candidates must rebase/freeze and rerun against the repaired baseline; their selective-CI evidence cannot waive this failure.
- Keep the change in this repository. If implementation unexpectedly requires a
community-basechange, stop and re-scope first; that shared package would require its own contract and both consumer suites perAGENTS.md.
Acceptance criteria
- The maintained Django test suite collects without importing
content_sync.tests.helpers,content_sync.dtc_content, orscripts.prod.sync_content_verify. - The implementation handoff contains a case-by-case mapping of the old verifier coverage to named current tests, newly ported tests, or a precise retired-only rationale; the old test is not simply skipped, renamed out of discovery, or deleted without that mapping.
- Current tests prove source-local create/update/delete, checksum/revision provenance, idempotence, invalid-input atomicity, and protection from deleting another source's rows.
- Current checkout-backed paths retain their existing dirty/revision/refusal and offline/no-network coverage; no test reaches GitHub or production data.
-
uv run --frozen python scripts/content.py --helpadvertises only executable, supported workflows, and every advertised subcommand has a focused dispatch/parser test. - Repository search finds no live code, tests, or operator instructions that invoke the removed verifier/helper. Historical audit prose may name it as history when clearly marked.
- No retired staged-import module, fixture tree, hardcoded public-content fallback, or checked-in projection is restored.
- Focused current sync/parser/command tests pass from synthetic local fixtures.
-
uv run --frozen python scripts/ci.py testcollects and passes in full; no exclusion, quarantine, or collection-ignore is added.
Repository and operations scenarios
- From a clean test environment, collect and run the focused sync/parser/command tests; they use synthetic fixtures and no network or operator staging tree.
- Exercise every
scripts/content.pysubcommand parser/dispatch contract and confirm none resolves a deleted file. - Run the full maintained Django tier and confirm collection reaches execution and passes.
- Inspect the coverage-disposition record against every former test class (
Agreement,Drift,MediaAsymmetry,RevisionStatus,Scope,Refusal,WorkingCopyIndependence,CheckoutPlan,ReadOnly,ReportShape,Convention). Reject unexplained loss.
Browser scenarios
Not applicable. This issue changes test/CLI wiring and directly affected operator documentation only; it must not alter a rendered route. Per _docs/PROCESS.md, the tester should classify screenshot evidence as not_applicable with this reason and run the applicable backend-only Playwright smoke tier selected by the frozen verification graph.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/tests/test_sync_content_verify.py, scripts/content.py, commit 1cd976a4, and the listed architecture, runbook, verification, and process documents. Run the focused sync/parser/command tests and inspect the current Django tests before mapping every former verifier case to a named equivalent, a ported test, or a retired-only rationale. Done means no dangling imports or advertised missing commands, required invariants remain covered, and uv run --frozen python scripts/ci.py test collects and passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, cli, documentation, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100