google / google/meridian

MCMC convergence catastrophically fails below a specific media channel count, independent of channel identity, priors, knots, precision, or version

Open
#1,778 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
297
Avg merge
1d 21h
Merged PRs (30d)
47

Description

Observed a hard convergence cliff: model converges cleanly with 23 media channels, fails catastrophically (max R-hat in the 10^12-10^14 range on beta_gm, negative R² at both geo and national level, posterior baseline contribution dropping from ~62% to ~18-25% of total KPI) at 22 channels or fewer.

Setup: 10 geos, 105 weekly time periods, 8 controls (all national-level constants, identical across geos per week), knots=20, max_lag=8, media_prior_type="roi", n_chains=4, n_adapt=5000, n_burnin=3000, n_keep=2000.

Isolation testing performed:

Tested removal of 3 different channel combinations at 22-21 channels; all failed identically regardless of which channels were removed
Confirmed the boundary precisely: 23 channels converges cleanly (all R-hat < 1.2, R² ~0.97 at geo level, no negative baseline probability flagged), 22 fails every time
Ruled out NaN propagation, spend duplication, and extreme spend/impression ratio outliers in the removed channels
Ruled out prior misconfiguration (reverted alpha prior changes, no effect)
Tested tightening eta_m/xi_c geo-hierarchy variance priors per your model-debugging docs (relevant since our controls are all national constants); no effect
Tested JAX backend with MERIDIAN_ENABLE_JAX_X64=true for 64-bit precision; hit a separate, unrelated dtype mismatch error (float64/float32 incompatibility in TFP bijector composition), consistent with issue #1163
Tested both v1.6.2 and v1.7.1; identical failure signature on both

Given the failure is reproducible across multiple different channel combinations, multiple prior configurations, two library versions, and is completely insensitive to which specific channels are removed, this looks like a structural issue tied to channel count itself rather than a data or configuration problem on our end.

Happy to provide a minimal reproduction script/dataset if useful.

Contributor guide

Open the contributing guide

Research direction

No source file, test, or entry point is identified in the issue. Start by reproducing the reported boundary with the promised minimal script or dataset, comparing 23 channels with 22 under the listed settings. Done means identifying the channel-count-dependent cause and establishing a regression test or confirmed fix for the convergence failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.