facebookexperimental / facebookexperimental/Robyn

Verifying Robyn predictions and optimization outputs

Open
#1,293 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.5k
Forks
433
PR merge metrics
No merged PRs in 30d

Description

I tried verifying Robyn predictions and faced some issues. This is how I went about verifying the predictions:

1) From the pareto-aggregated file, I fetched the coefficients.
2) The media transformation flow is - raw spends --> adstock function (geometric, in my case) --> hill function.
3) Fetched the adstocked media from the media_transformation. In the column names type, I could see adstockedMedia and decompMedia. I went ahead with adstockedMedia.
4) For applying, hill saturation, I fetched the alphas and gammas from the hyperparameters file.
5) After applying hill transformation, I simply did coefficients*transformed media.

I could not reproduce the Actual v Predicted plot.

The reason why I want to verify Robyn outputs is because I see some discrepancy in the contribution chart and the initial total response in the budget allocator output.

The total estimated KPI is 18,80,259. In the contribution screenshot below, the media contributes 7.7% to the KPI (144780) whereas in the budget allocator result , the initial total response is 2,30,000, which is 12.23% of the estimated KPI.

Image

Image

So my questions are as follows:

1) Is the above mentioned discrepancy expected?
2) Is my approach to verifying Robyn predictions, as mentioned above, correct?
3) In your practice, do you see media contributing as less as 7.7%?

Lastly, apologies if questions like these are supposed to be asked elsewhere. If so, please let me know where I can post this. Thanks in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.