PostHog / PostHog/posthog

Prod-ready realtime behavioral cohorts

Open
#80,291 1 comment 0 reactions 2 assignees View on GitHub

@gustavohstrassburger is already working on this.

Since Aug 10, 2026.

feature/cohorts feature/feature-flags team/feature-flags
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Where we are

The streaming pipeline (shuffler, processor, seeder) is deployed on current master and healthy, ingesting events for allowlisted teams and producing membership transitions to the shadow topic.
The backfill machinery is fully built and merged: seed topic, run and chunk tables, seeder scans, fenced and paced seed apply, reconcile snapshots with a dedicated marker topic, kind aware completion, and the readiness finalizer.
Backfill runs work end to end but are operator driven from a toolbox pod, and they park short of completion: behavioral runs stop at reconciling because the Django finalizer is disabled, person property runs stop at seeding because person reconcile dispatch is off in the seeder.
The save triggered dispatch path is fully built but dark: the Django fleet has none of the cohort env vars set (realtime allowlist, trigger allowlist, attestations).
The processor PVC has been resized to 150GiB.

Next steps

  1. Ops hardening. Arm the processor's storage guards: COHORT_PERSON_RECORD_TTL_DAYS so the person record store stops growing unboundedly, and COHORT_SEED_DISK_PAUSE_PCT so seeding pauses itself before disk pressure. Add a first set of alert rules covering stalled runs, failed chunks, disk utilization, produce errors, and trigger outcomes, since with automatic triggering nobody will be watching runs by hand. Also explain the recent person seed skips by reason and recreate any run that lost seeds.

  2. One time run cleanup. Produce an inventory of every backfill run in an active status across all teams, terminalize the stale ones, and write down the short list of verified runs the finalizer is allowed to stamp. This must land before the finalizer is enabled, because the finalizer stamps readiness for anything that qualifies the moment it turns on and stamps are one way.

  3. Django enablement. Ship the standing env on web and workers: realtime team allowlist, the merge gate and durability attestations, a person topic bytes budget, a longer default person scan horizon, and the finalizer flag. Success looks like the parked behavioral runs reaching completed and stamping readiness within minutes of rollout. Then run resave_cohorts for the allowlisted teams and verify every realtime cohort has a non null condition_type, since stale classification silently breaks behavioral evaluation.

  4. Person run completion. Make person property runs travel the same path as behavioral ones. Flip the flags membership stamp policy in every region first (inert while flags reads are off, but the ordering matters), then enable person reconcile dispatch in the seeder so parked person runs get reconciled and observed, then enable the person readiness gate so the finalizer completes and stamps them.

  5. Automatic triggering. Enable the trigger allowlist for a single pilot team so that saving a behavioral cohort creates, seeds, reconciles, and completes a backfill run with no human involvement. Prove it with a hands off cohort edit end to end, then soak for a week on the alert set before widening the allowlist to more teams.

Out of scope for this milestone: pointing membership output at the topic CDP consumes, and the flags read path.
Small code fixes recommended alongside: a catalog refresh staleness metric (a silently stale filter catalog during a run is invisible today), failing a run when a chunk exhausts its retries (today it parks in seeding forever and blocks future runs for that cohort), and splitting the trigger refusal counter so budget refusals and blocked run slots are distinguishable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.