PolicyEngine / PolicyEngine/boe-var-model
Predictive bands under-cover: add stochastic volatility or importance-weighted posterior predictive bands
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
The committed coverage study (results/coverage_evaluation.json, code_version 826619a2cd445bc6bc0d3a540d81ca26e907118f0faed446a3b4f347d7b73833; 49 expanding-window origins, h1–h8, gated by tests/test_committed_artifacts.py) shows the predictive bands run materially too narrow, worsening with horizon:
- uk_gdp: 68% band covers 0.878 at h1 falling to 0.531 at h8; 90% band 0.918 → 0.755.
- cpisa: 68% band covers 0.531 at h1 falling to 0.367 at h8; 90% band 0.776 → 0.592.
- Overall the 68% bands contain outturns ~62% of the time and the 90% bands ~77%.
The current bands come from a homoskedastic reduced-form BVAR posterior predictive (posterior draws × Gaussian shock paths), which cannot represent the volatility clustering (2008, 2020–22) in the sample, so the tails are too thin at every horizon.
Proposed model-side fix, either of:
- Stochastic volatility in the shock process (e.g. Primiceri-style SV, or a common volatility factor a la Carriero–Clark–Marcellino), so predictive dispersion widens where the data demand it; or
- Importance-weighted posterior predictive bands: re-weight predictive draws (or scale shock variances) using the realised pseudo-out-of-sample forecast-error distribution, i.e. a nonparametric calibration of band quantiles against the errors already computed for the coverage study.
Either should be validated by re-running the same expanding-window coverage evaluation and committing the improved coverage numbers.
Interim note: the PolicyEngine/macro site now applies an empirical correction at chart render time — band half-widths on the published /forecasts fans are scaled by k(h, level) = z(nominal)/z(empirical) derived from this repo's coverage_evaluation.json (committed as forecasts/band_calibration.json there; archived round artifacts untouched). A model-side fix would make that correction unnecessary.
🤖 Generated with Claude Code
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with results/coverage_evaluation.json and tests/test_committed_artifacts.py to understand the committed expanding-window evaluation and its current gates. Compare the proposed stochastic-volatility and importance-weighted approaches against the existing predictive-band workflow, then rerun the same evaluation. Done means the model-side bands improve the reported coverage and the updated coverage numbers are committed, without relying on forecasts/band_calibration.json.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100