Funnel sampling inherently changes results
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 7h 27m
- Merged PRs (30d)
- 222
Description
Bug Description
We are sampling by distinct_id. If a customer has a bunch of person merges, it can ruin their funnel sampling.
If they are aggregating on anything other than unique_user, it will be off too.
Accurate sampling for a funnel has to "sample on" the same target that the funnel is running over. Unfortunately clickhouse doesn't support this - the sampling key has to be defined as part of the table.
Debug info
Thread here: https://posthog.slack.com/archives/C082C0UNW4X/p1758828001064339
Original sampling issue: https://github.com/PostHog/posthog/issues/12908#issuecomment-1433573133
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source files, tests, or entry points are named. Start with the linked Slack thread and original sampling issue, then identify the funnel sampling path and relevant ClickHouse table configuration; the work is only ready when an implementable correction and its validation criteria are defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse
- Domain
- analytics, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100