facebook / facebook/prophet

Credible intervals when using Prophet without MCMC are not symetric nor equal.

Open
#2,127 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
20.4k
Forks
4.6k
Avg merge
19h 52m
Merged PRs (30d)
1

Description

Hi,

I have realized something is happening with the credible intervals obtained by Prophet. In the point estimate case, ie when the parameters are learned and not sampled using MCMC, then the probability of the regressed variable given the different prophet models has the following form:

N( Y(t) | g(t) + s(t) + h(t), \sigma^2)

In other words, the regressed variable follows a Gaussian distribution with mean given by the prophet models (trend, seasonality, holidays) g(t) + s(t) + h(t), and a variance of \sigma^2. In the point estimate case the parameters of g(t), s(t) and h(t) are fixed, which means that all the randomness comes from \sigma^2. Or in other words, e.g. the 95% credible intervals should be at +- 2 \sigma; and any credible interval should be symmetric around the mean, since the Gaussian distribution is symetric.

I have observed this is not true in practice. By looking at the source code, Prophet estimates this credible intervals by sampling from N( Y(t) | g(t) + s(t) + h(t), \sigma^2) and then computing percentiles. So, beyond small numerical errors, I should expect the credible intervals to be symmetric. For example using prophet's dataset `example_wp_log_peyton_manning.csv` I obtain the following credible intervals. I will print the first 3 forecast of `yhat`. In practical terms this implies running:
```
forecast = model.predict(df)

yhat_lower = forecast['yhat_lower']
yhat_upper = forecast['yhat_upper']
yhat = forecast['yhat']

print(yhat_upper[0:3]-yhat[0:3])
print(yhat[0:3]-yhat_lower[0:3])
```
and what I obtain is:

Difference between upper interval and median
0 1.007656
1 0.915107
2 0.891538
dtype: float64
Difference between median and lower interval
0 0.855754
1 0.905877
2 0.900042

we can see two things: the first is that credible intervals change across time and this is not to be expected since from what I have understood the noise parameter is a single parameter; and this can be checked by printing `print(model.params['sigma_obs'])`. The second is that the lower and upper credible interval probability mass are not equal, and we should expect it to be since the Gaussian is symetric.

Any thought on what might be happening?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.