Measure the onboarding funnel and split the cohorts
- Dominant language
- JavaScript
- Stars
- 400
- Forks
- 89
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 146
Description
Two pieces: instrumenting the onboarding path so we can see where people fall out of it, and splitting new signups into the new path and the old one so the comparison means something.
### Worth knowing
The before-number is contested, and that's worth knowing up front rather than discovering it in a review. There are at least three figures in circulation and none of them is verified: roughly 17% of signups opening an instance editor within 14 days, product#557's ~22.3% activation to first instance, and product#558's ~5% building a first application. Those two issues exist precisely because nobody trusts the numbers. So whatever we measure here, expect the baseline to be argued about, and it's worth agreeing which number we're moving before we claim to have moved it.
The cohort split has a constraint that shapes it. The decision about whether to auto-provision used to happen at email verification, server-side, when no browser is running. And `forge/product/index.js` only decorates `capture`, so there's no server-side PostHog flag evaluation available even though the library supports it. That's why the split works by turning auto-provisioning off for everyone on the flagged deployment and having the control cohort call the on-demand provisioning endpoint on arrival instead. Assignment stays entirely in the browser and forge needs no new capability.
`posthogFlags` already exists on the `account-settings` store, so flags are surfaced to the frontend today.
Contributor guide
Assessment
This issue has not been assessed yet.