DataTalksClub / DataTalksClub/website

Restore green CI on main: stale podwiki count pins in the content-update contract test

Open
#303 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug content infra operations P0 testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Raw user intake — needs PM grooming. main is currently RED.

Symptom

CI run https://github.com/DataTalksClub/website/actions/runs/33621318177 for 55f6743 concluded cancelled. The quality job failed and cancelled the run; ci-gate then failed with QUALITY_RESULT=cancelled DJANGO_RESULT=cancelled PLAYWRIGHT_RESULT=cancelled.

Failing gate: qualitymake test-ciuv run --frozen pytest ci/tests tests_ci -q (Makefile:192).

ci/tests/test_content_update.py:69: AssertionError
E   {"search_documents": 2996} != {"search_documents": 2998}
E   {"graph_nodes": 1070}      != {"graph_nodes": 1072}
E   {"graph_links": 12987}     != {"graph_links": 13006}
1 failed, 616 passed in 40.00s

Reproduces locally at 55f6743. Not a flake.

Root cause

The stale pins are the podwiki block at ci/tests/test_content_update.py:56-61. Counts are computed deterministically from the checked-in projection by _family_counts (ci/content_update.py:375).

Two lanes crossed:

  • 53eef66 "Add unified content update CI contract" introduced the test with pins correct at that time.
  • 4b6187f "Refresh projection for organized content source" de-duplicated the podcast catalogue during route canonicalization, removing two aliased nodes (podcast:s12e08, podcast:theme-park-crowd-modeling-to-tesla-full-stack-data-engineering) and 19 graph links from content/public_projection/wiki_graph.json and wiki_search.json. It correctly refreshed five other pinned contracts but not this newly-added one.

Measured across the push:

face8e4  links=13006 nodes=1072 search=2998 wiki=282
53eef66  links=13006 nodes=1072 search=2998 wiki=282   <- pins authored here
4b6187f  links=12987 nodes=1070 search=2996 wiki=282   <- break introduced here
55f6743  links=12987 nodes=1070 search=2996 wiki=282

The projection change is intended. Only the pin is stale.

Proposed fix (PM to confirm)

Update the three integers at ci/tests/test_content_update.py:57-59 to graph_links: 12987, graph_nodes: 1070, search_documents: 2996, leaving wiki_pages: 282.

This is explicitly NOT the #302/#253 re-pinning hazard: these are plain aggregate counts asserting redaction-safe report shape, not a content-authority digest, and the drift they caught is an intended de-duplication. Grooming should still confirm that reasoning.

Design question worth deciding here

A hardcoded exact-count pin inside ci/tests/ will red-line main on every legitimate content refresh — working against the purpose of the content-update pipeline the same commit introduced. Consider asserting invariants (non-zero, bounded, consistent with manifest.json) rather than exact literals. PM to decide whether that belongs in this issue or a follow-up.

Missing evidence — do not assume the fix is sufficient

django and playwright were killed at ~2 minutes by the quality job cancelling the run (.github/workflows/ci.yml:499-503). So make test-factories, make test-migrations, make test-django-full and the full Playwright matrix have no verdict for 55f6743.

The pre-push scheduled baseline (run 33591717271) failed migrations on content/tests/test_editorial_route_migration_contract.py:48 (required_content_sha256 mismatch). 4b6187f edited exactly that test, so it may be resolved — unverified. Verification must produce fresh full Django and Playwright evidence; do not rerun failed jobs on the cancelled run.

Related, but separate

  • #302 — the inventory validator is red independently. Explicitly ruled out as the cause here: validate_github_editorial_source_projection_inventory.py was not among this run.s selected quality targets. Confirmed still failing at both face8e4 and 55f6743, i.e. pre-existing.
  • #253 — the projection reproducibility reconciliation.

Process note

31 of the 50 commits in face8e4..55f6743, including 4b6187f and 53eef66, carry no Closes #N / Refs #N trailer, contrary to _docs/PROCESS.md step 6. No owning issue existed for the content-update CI contract or the projection reorganization, which is why this issue is being filed rather than an existing one reopened.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ci/tests/test_content_update.py and ci/content_update.py:_family_counts to understand the pinned report shape, then inspect Makefile:192 and .github/workflows/ci.yml:499-503. Confirm whether this issue should only refresh the three podwiki counts or also address invariant-based assertions. Done means the quality tests pass with the intended counts and fresh Django and Playwright results are recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, github-actions, playwright, python
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.