Feature Request: Support experiments without a control variant (0% control use case)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 7h 27m
- Merged PRs (30d)
- 222
Description
Feature request
Is your feature request related to a problem?
Some teams run experiments iteratively and would like to compare multiple test variants without requiring a control group.
Right now, PostHog experiments require at least one variant to be marked as "control" and to have exposure events in order to calculate conversion metrics. If the control group is set to 0%, results will not show, even if test variants are receiving traffic.
Describe the solution you'd like
We’d like to run experiments with:
- 0% control and 100% split across test variants
- Full metric reporting and statistical significance between non-control variants
- Ability to designate "control" as optional, or allow users to re-assign a previous variant as baseline without creating a new experiment
This would support workflows where:
- A previously tested variant becomes the new default
- Teams want to test improvements to that new baseline
- Avoids having to reset the feature flag or re-implement tracking
Describe alternatives you've considered
- Creating a new feature flag and starting a fresh experiment (adds implementation overhead)
- Reusing the same flag but manually analyzing results with HogQL (non-trivial for many teams)
- Temporarily assigning a “dummy” control variant at 1% (skews metrics and complicates interpretation)
Additional context
From: https://posthoghelp.zendesk.com/agent/tickets/33830 (moved to PostHog: https://us.posthog.com/project/2/support/tickets/35341)
Debug info
null
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by tracing how experiments require a control variant and how conversion metrics and statistical significance are calculated. Done means supporting a 0% control split, reporting metrics for test variants, and comparing non-control variants without requiring a new experiment.
Written by the indexing model from the issue text.
Assessment
- Domain
- analytics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100