dwyl / dwyl/learn-google-optimize

Multi-arm bandit no longer applies

Open
#10 0 comments 1 reaction 0 assignees View on GitHub
external-dependency
Dominant language
HTML
Stars
23
Forks
4
PR merge metrics
No merged PRs in 30d

Description

I'm removing the text about the multi-armed bandit because Optimize doesn't include it. However, I did find mention of there being an Optimize version in the pipeline so I'll leave the old text here for future us:

+ **Watch out for the [multi-armed bandit](https://support.google.com/analytics/answer/2844870?hl=en&ref_topic=1745207)**!
This option is _on_ by default but you can **switch it off** in CE by ensuring `'Distribute traffic evenly across all variations'` is switched on in the Advanced options
![switch-off-multi-armed-bandit](https://cloud.githubusercontent.com/assets/4185328/10583984/cecf67ba-7687-11e5-9d1d-57f96faea3bd.png)

+ The _multi-armed bandit_ is a method that CE uses, which looks at your experiment twice a day and **redirects a slightly greater proportion of your traffic** to the variation that appears to be more successful
+ In theory, this provides you with a more successful overall experiment because you don't have to 'wait until the end of the experiment' to implement your winning strategy
+ There are [many critics of this algorithm](https://brianclifton.com/blog/2013/09/24/google-content-experiments/) as it can lead to skewed results if the data is not analysed correctly
+ But there are also some companies that have [built their business on this model](https://mynaweb.com/) as it can also be an efficient testing tool
+ We **recommend you switch this off if this is your first experiment** so you can get a cleaner feel for the results, but don't forget to play with it later on!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.