Credible intervals when using Prophet without MCMC are not symetric nor equal.
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 4.6k
- Avg merge
- 19h 52m
- Merged PRs (30d)
- 1
Description
Hi,
I have realized something is happening with the credible intervals obtained by Prophet. In the point estimate case, ie when the parameters are learned and not sampled using MCMC, then the probability of the regressed variable given the different prophet models has the following form:
N( Y(t) | g(t) + s(t) + h(t), \sigma^2)
In other words, the regressed variable follows a Gaussian distribution with mean given by the prophet models (trend, seasonality, holidays) g(t) + s(t) + h(t), and a variance of \sigma^2. In the point estimate case the parameters of g(t), s(t) and h(t) are fixed, which means that all the randomness comes from \sigma^2. Or in other words, e.g. the 95% credible intervals should be at +- 2 \sigma; and any credible interval should be symmetric around the mean, since the Gaussian distribution is symetric.
I have observed this is not true in practice. By looking at the source code, Prophet estimates this credible intervals by sampling from N( Y(t) | g(t) + s(t) + h(t), \sigma^2) and then computing percentiles. So, beyond small numerical errors, I should expect the credible intervals to be symmetric. For example using prophet's dataset `example_wp_log_peyton_manning.csv` I obtain the following credible intervals. I will print the first 3 forecast of `yhat`. In practical terms this implies running:
```
forecast = model.predict(df)
yhat_lower = forecast['yhat_lower']
yhat_upper = forecast['yhat_upper']
yhat = forecast['yhat']
print(yhat_upper[0:3]-yhat[0:3])
print(yhat[0:3]-yhat_lower[0:3])
```
and what I obtain is:
Difference between upper interval and median
0 1.007656
1 0.915107
2 0.891538
dtype: float64
Difference between median and lower interval
0 0.855754
1 0.905877
2 0.900042
we can see two things: the first is that credible intervals change across time and this is not to be expected since from what I have understood the noise parameter is a single parameter; and this can be checked by printing `print(model.params['sigma_obs'])`. The second is that the lower and upper credible interval probability mass are not equal, and we should expect it to be since the Gaussian is symetric.
Any thought on what might be happening?
Contributor guide
Assessment
This issue has not been assessed yet.