PostHog / PostHog/posthog

Isolate products/workflows for selective CI and move remaining team code out of posthog/

Open
#84,402 0 comments 0 reactions 1 assignee View on GitHub

@mayteio is already working on this.

Since Aug 18, 2026.

team/workflows
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Problem

Every workflows-team PR runs the full backend suite. products/workflows, products/cdp, and products/messaging are not isolated products: none declares backend:contract-check, so turbo-discover cannot skip the full Django suite for them. The merge queue uses the same selection, so this cost lands on every Trunk run.

Some team-owned code also still lives under posthog/ (posthog/cdp/, messaging leftovers), where any change always triggers the full suite.

Goal

Move everything movable into products/, and isolate workflows (then messaging) so internal-only changes skip the full suite. Follow the isolating-product-facade-contracts skill and products/architecture.md. Use products/error_tracking as the reference implementation.

Recon (2026-08-18)

  • hogli product:isolate:scan workflows: core-coupling count 35 (single-PR territory per the skill).
  • hogli product:isolate:scan messaging: core-coupling count 10.
  • hogli product:maturity workflows: 51/100. Models are fully in-product. There is no real facade. Views live in backend/api/ with 42 direct .objects calls. api/hog_flow.py is ~4,600 lines and most of it is business logic, not HTTP.
  • Wiring is nearly free: no Temporal code, no query runners, Celery tasks already in backend/tasks/, max_tools.py already at the designated path.
  • Sequencing constraint: creating facade/contracts.py flips hogli product:lint to strict mode, which fails on the non-canonical dirs backend/{api,providers,services,utils}/. Structure must become canonical before or with the facade.

Track W: isolate products/workflows

  • W1: fold non-canonical dirs into logic/. Move backend/providers/, backend/services/, and backend/utils/ under backend/logic/. Rewrite the ~10 external dotted references (posthog/models/integration.py, posthog/tasks/integrations.py, posthog/api/integration.py, customer_analytics, ai_observability) and the @patch strings in posthog/models/test/test_integration_model.py. Add the root tsconfig.json. Lenient lint tolerates this, so it can land first.
  • W2: move backend/api/ to backend/presentation/. Run hogli product:isolate:move workflows (dry-run first). It moves viewsets to presentation/views/, relocates api tests to backend/tests/api/, pins Celery task names, and rewrites dotted paths repo-wide. Reroute the two posthog/urls.py bypasses (SES events webhook, PublicHogFlowTemplateViewSet) through backend/routes.py. Thick views get ignore_imports TODO entries in pyproject.toml. This PR conflicts with every open workflows PR, so land it in a quiet window and regenerate on conflict.
  • W3: facade + contracts + core-caller sweep. Add backend/facade/contracts.py (frozen pydantic dataclasses) and backend/facade/api.py with explicit mappers, plus facade/tasks.py re-exporting the two SES beat tasks. Sweep the 35 core references from the scan: HogFlow and TeamWorkflowsConfig model access becomes capability functions returning contracts, provider and integration-usage callers move to facade functions, test fixtures switch to apps.get_model with TYPE_CHECKING imports. Reroute the dotted HogFlow.State reference in posthog/settings/web.py to a facade path. Replace the 6 cross-product imports of products.cdp...test_hog_function_templates.MOCK_NODE_TEMPLATES with a non-test fixture home. Add workflows to the shared [[interfaces]] regex in tach.toml. Note: workflows cannot use a MODEL_CROSSINGS allowance (that list only shrinks), so HogFlow never crosses as a class.
  • W4: presentation wave. Thin HogFlowViewSet (~1,900 lines, 21 actions): extract the perform_update write semantics (draft routing, staleness fences, select_for_update, revision bump, reschedule), secret partitioning and rehydration, publish-impact and token minting, and template resolution into logic/ behind facade functions, with parity tests. Do the same, smaller, for HogFlowTemplateViewSet. Done means zero workflows ignore_imports entries remain.
  • W5: seal and enable. Add backend:contract-check to package.json and a products/workflows/turbo.json with inputs narrowed to backend/facade/**, backend/presentation/**, backend/routes.py, backend/tasks/**, backend/max_tools.py. hogli product:lint blocks this until W4 empties the allowlist. This is where the CI skip turns on.

Track M: isolate products/messaging

  • Same recipe at coupling 10: restructure non-canonical dirs, build a facade for the MessageCategory / preferences / customerio sync / remote-config consumers, reroute the two posthog/urls.py webhook imports, sweep. Workflows imports messaging internals from api/hog_flow.py (design operations, unlayer, MessageTemplate), so messaging's facade should land before or with W3. Likely 1-2 PRs.

Track P: move posthog/ leftovers into products/

  • P1: move posthog/models/messaging.py, posthog/models/message_assets/, posthog/models/hog_invocation_results/, and posthog/api/hog_invocation_*.py into the owning products. Keep tables via SeparateDatabaseAndState.
  • P2: move posthog/cdp/ into products/cdp/backend/ (~17.7k LOC: ~12.9k templates, ~4.7k core including validation.py and filters.py that workflows depends on). Mechanical but wide: 129 importing files. This is a move only, not isolation.
  • P3 (optional): move frontend/src/scenes/data-pipelines and frontend/src/scenes/hog-functions into product frontends. Organization win, no backend-CI effect.

Out of scope

  • cdp isolation. 75 inbound importers, with Plugin/PluginConfig used throughout core. Needs its own scan after P2 and likely the facade-first, team-sliced path. Plan separately.
  • The integration platform (posthog/models/integration.py and friends, 344 importers). Shared platform infrastructure used by many teams; it stays in posthog/.
  • nodejs/src/cdp. Separate service, orthogonal to the Django CI matrix.

Verification

  • Per PR: tach check --dependencies --interfaces, lint-imports, hogli product:lint workflows, hogli product:maturity workflows.
  • After W5: open a trivial workflows-internals PR and confirm turbo-discover reports run_legacy=false with only workflows in the matrix.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.