humanmade / humanmade/experiments

Add target condition or percentage "lift" to A/B test success criteria

Open
#23 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
11
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Right now just achieving a statistically significant increase in a conversion rate doesn't really account for the desired amount of "lift" we might want.

A small increase in a value may not actually be enough to want to make changes on the back of.

This target lift should give us enough information to calculate a target sample size before the test starts if we're able to query the current baseline via elastic. This sample size should allow us to estimate a recommended amount of time to run the test for.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.