MCMC convergence catastrophically fails below a specific media channel count, independent of channel identity, priors, knots, precision, or version
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 297
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 47
Description
Observed a hard convergence cliff: model converges cleanly with 23 media channels, fails catastrophically (max R-hat in the 10^12-10^14 range on beta_gm, negative R² at both geo and national level, posterior baseline contribution dropping from ~62% to ~18-25% of total KPI) at 22 channels or fewer.
Setup: 10 geos, 105 weekly time periods, 8 controls (all national-level constants, identical across geos per week), knots=20, max_lag=8, media_prior_type="roi", n_chains=4, n_adapt=5000, n_burnin=3000, n_keep=2000.
Isolation testing performed:
Tested removal of 3 different channel combinations at 22-21 channels; all failed identically regardless of which channels were removed
Confirmed the boundary precisely: 23 channels converges cleanly (all R-hat < 1.2, R² ~0.97 at geo level, no negative baseline probability flagged), 22 fails every time
Ruled out NaN propagation, spend duplication, and extreme spend/impression ratio outliers in the removed channels
Ruled out prior misconfiguration (reverted alpha prior changes, no effect)
Tested tightening eta_m/xi_c geo-hierarchy variance priors per your model-debugging docs (relevant since our controls are all national constants); no effect
Tested JAX backend with MERIDIAN_ENABLE_JAX_X64=true for 64-bit precision; hit a separate, unrelated dtype mismatch error (float64/float32 incompatibility in TFP bijector composition), consistent with issue #1163
Tested both v1.6.2 and v1.7.1; identical failure signature on both
Given the failure is reproducible across multiple different channel combinations, multiple prior configurations, two library versions, and is completely insensitive to which specific channels are removed, this looks like a structural issue tied to channel count itself rather than a data or configuration problem on our end.
Happy to provide a minimal reproduction script/dataset if useful.
Contributor guide
Research direction
No source file, test, or entry point is identified in the issue. Start by reproducing the reported boundary with the promised minimal script or dataset, comparing 23 channels with 22 under the listed settings. Done means identifying the channel-count-dependent cause and establishing a regression test or confirmed fix for the convergence failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100