PolicyEngine / PolicyEngine/boe-var-model

Predictive bands under-cover: add stochastic volatility or importance-weighted posterior predictive bands

Open
#12 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
1
Avg merge
13m
Merged PRs (30d)
1

Description

The committed coverage study (results/coverage_evaluation.json, code_version 826619a2cd445bc6bc0d3a540d81ca26e907118f0faed446a3b4f347d7b73833; 49 expanding-window origins, h1–h8, gated by tests/test_committed_artifacts.py) shows the predictive bands run materially too narrow, worsening with horizon:

  • uk_gdp: 68% band covers 0.878 at h1 falling to 0.531 at h8; 90% band 0.918 → 0.755.
  • cpisa: 68% band covers 0.531 at h1 falling to 0.367 at h8; 90% band 0.776 → 0.592.
  • Overall the 68% bands contain outturns ~62% of the time and the 90% bands ~77%.

The current bands come from a homoskedastic reduced-form BVAR posterior predictive (posterior draws × Gaussian shock paths), which cannot represent the volatility clustering (2008, 2020–22) in the sample, so the tails are too thin at every horizon.

Proposed model-side fix, either of:

  1. Stochastic volatility in the shock process (e.g. Primiceri-style SV, or a common volatility factor a la Carriero–Clark–Marcellino), so predictive dispersion widens where the data demand it; or
  2. Importance-weighted posterior predictive bands: re-weight predictive draws (or scale shock variances) using the realised pseudo-out-of-sample forecast-error distribution, i.e. a nonparametric calibration of band quantiles against the errors already computed for the coverage study.

Either should be validated by re-running the same expanding-window coverage evaluation and committing the improved coverage numbers.

Interim note: the PolicyEngine/macro site now applies an empirical correction at chart render time — band half-widths on the published /forecasts fans are scaled by k(h, level) = z(nominal)/z(empirical) derived from this repo's coverage_evaluation.json (committed as forecasts/band_calibration.json there; archived round artifacts untouched). A model-side fix would make that correction unnecessary.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with results/coverage_evaluation.json and tests/test_committed_artifacts.py to understand the committed expanding-window evaluation and its current gates. Compare the proposed stochastic-volatility and importance-weighted approaches against the existing predictive-band workflow, then rerun the same evaluation. Done means the model-side bands improve the reported coverage and the updated coverage numbers are committed, without relying on forecasts/band_calibration.json.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.