DataTalksClub / DataTalksClub/website
D1.1 Replace DTC jobs with the package jobs app (relay backend)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Plan issue: community-base docs/plan/phase-1.md D1.1. Depends on C1.5, R1.1, R1.2 (all done); the D0.1a package pin (v0.3.0) is on main.
Goal
Replace the site's jobs app (DurableJob machinery on django_q) with the community-base jobs app, executed by Relay.
Scope
- Install
community_base.jobs(P2). Re-point everyjobs.registry.register_handlerandjobs.dispatchcall site to the package, keeping handler names. - Move schedule semantics to the package: the package's
cb-jobs-run-due/cb-jobs-sweepRelay schedules drive due intents; the site'sdtc:durable-job-relaydjango_q bridge dies with django_q. - Data migration (P6): copy pending
jobs.DurableJobrows intocb_jobs.JobIntent(raw-SQL migration in a surviving app so it runs after the old app is removed); old tables stay for a later cleanup. - Settings:
COMMUNITY_BASE["SITE_URL"],RELAY_BASE_URL,RELAY_API_KEY,RELAY_WEBHOOK_SECRETread from the environment (deployment secrets provision the values);JOBS_BACKEND="relay"in development/production,"sync"in test/local. - Add
sync_relay_schedulesto the deploy steps after migrate; remove theqclustercontainer from deploy task definitions. - Remove
django_qfromINSTALLED_APPS(+Q_CLUSTER), remove thejobs/app and its tests; mount the package ingress route and allowinternal/jobs/runin thecore.middlewarerequest boundary (CSRF exempt, no session).
Non-goals
- Handler business logic does not change — only registration and transport.
- Dropping the orphaned
jobs_durablejob/ django_q tables (later cleanup once the copy is verified in development). - Provisioning the Relay client secret in the dev environment (operator step; the code reads
RELAY_API_KEYfrom env and dispatch failures are retried until it is set).
Acceptance criteria
-
uv run pytest -qpasses (full suite) -
uv run python manage.py makemigrations --check --dry-runreports no changes beyond the intended migration -
grep -rn "django_q\|from jobs" --include=*.py website/ events/ email_app/ content_sync/ api/shows no stale runtime imports outside the historical migration - Development deploy:
manage.py jobs_ingress_selftest-> OK andsync_relay_schedules --dry-run-> no diff (needs the operator-provisioned Relay client secret; recorded here when run)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the D1.1 plan reference in docs/plan/phase-1.md, then trace jobs.registry.register_handler and jobs.dispatch call sites, the jobs app, settings, deploy task definitions, and core.middleware. Start by mapping the package integration and raw-SQL data migration dependencies. Done means the full pytest and migration checks pass, no stale runtime imports remain, and the deployment self-test and schedule sync meet the stated acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, database, devops
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100