facebookexperimental / facebookexperimental/Robyn

Areas for Improvement When Using Robyn

Open
#1,098 1 comment 0 reactions 1 assignee Claimed by @gufengzhou View on GitHub
Dominant language
Jupyter Notebook
Stars
1.5k
Forks
433
PR merge metrics
No merged PRs in 30d

Description

Hello :)
I have been using Robyn quite effectively, but I have noticed a few areas for improvement that I would like to share.

1. **Issue with Budget Allocator Onepager Results**:

- The current values focus solely on paid media, leading to a significant gap with actual sales (Nonpaid + Paid). When collaborating with marketing practitioners, I can only explain that "The total response value is not the expected actual sales, as the impact of Non_paid_media is not considered, so the real value may be higher or lower." This explanation is not intuitive and is difficult to justify.

2. **Baseline Interpretation Issue**
- When defining Sales = Paid Media + Non Paid Media(=baseline), it becomes challenging to interpret if the baseline is negative. For example, when Sales is 1,000,000, Robyn may express this as 2,500,000 (Paid Media) - 1,500,000 (Non Paid Media). It would be beneficial to ensure that Non Paid Media does not become negative.

3. **Lack of Forecasting Features**
- There is a need for intuitive functionality, such as estimating "What will the expected sales be if the budget allocation strategy is used for 3 months?" I understand that Recast's MMM already includes this feature.

5. **Lack of Validation Functionality**
- To continually trust the model, tracking the error rate over time for previously used models is essential. A method to visually compare the total response value from the Budget allocator with the actual values after a simulation period using a time-series graph would be helpful. Additionally, separating validation for NonPaid Media and Paid Media could be beneficial. Although I understand that predicting NonPaid Media is challenging, having a rough estimate would enhance intuitive expression.

7. **Urgent Need for Refresh Performance Improvement**
- I wish to refresh the model weekly to observe changes in channel contributions, but the refresh process is unstable. For instance, if the Initial Model shows a 50% contribution from Campaign A, the Refreshed Model may drastically decrease to 20% one week later. The inconsistency and instability make it challenging to use the model continuously. Simply selecting the model with the lowest decomp value as the final model does not resolve the issue.

I would like to know if any of the issues mentioned above are currently being developed. I am also curious about the scheduled updates. Please let me know if any part is unclear.
Thank you :)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.