lnccbrown / lnccbrown/HSSM

[Migration] Regenerate the recorded trace fixtures that predate bambi 0.20 (*_Intercept_centered)

Open
#1,336 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
124
Forks
24
Avg merge
19h 32m
Merged PRs (30d)
60

Description

Part of #1305. Parent: #1306.

⚠️ **Found while fixing F6 (#1315)** — it was masked by R10. Removing the R10 xfail marks on the 16 `tests/test_plotting_cartoon.py` ids made them fail one step later, and the same gap explains why the R2-marked `test_sample_posterior_predictive` grid still fails after the prediction path was fixed.

From bambi 0.20's *Intercept handling* (see F12, #1330): every common intercept is now a `*_Intercept_centered` free RV with the uncentered `*_Intercept` kept as a deterministic. HSSM keeps the centered RV in `posterior` because bambi's `predict` and `compute_log_likelihood` (via `pm.compute_deterministics`) raise `KeyError` without it.

The recorded trace fixtures under `tests/fixtures/` predate that change and only carry `*_Intercept`:

| fixture | loaded by | missing |
|---|---|---|
| `cavanagh_idata.nc` | `cav_dt` (`tests/conftest.py`) | `v_Intercept_centered` |
| `idata_cavanagh_cartoon.nc` | `cav_model_cartoon` | `v_Intercept_centered`, `a_Intercept_centered` |
| `test_idata_race.nc` | `race_model_cartoon` | `v0_Intercept_centered`, `a_Intercept_centered` |
| `idata_race_cartoon.nc` | (no test references it) | `v0_Intercept_centered`, `a_Intercept_centered` |

```
KeyError: 'v_Intercept_centered'
src/hssm/plotting/model_cartoon.py:397 compute_merge_necessary_deterministics
pymc/sampling/deterministic.py:176 compute_deterministics
```

Two consequences today:

- Any in-sample `Model.predict` / `compute_deterministics` on these traces raises `KeyError` (the cartoon tests, the `test_sample_posterior_predictive` grid).
- Out-of-sample `predict(data=...)` on them does **not** raise: `pm.sample_posterior_predictive` silently resamples the missing RV from its prior, so the quantile-probability and predictive plotting tests pass on draws that are partly prior, not posterior.

### Tasks
- [ ] Regenerate the trace fixtures with the bambi 0.20 model definitions used by their fixtures (`cav_model_cartoon`, `race_model_cartoon`, the `v ~ theta + (1|participant_id)` model behind `cav_dt`), keeping draw counts small; decide whether `idata_race_cartoon.nc` is dead and can be deleted
- [ ] Add a regeneration script (or document the recipe) so the fixtures can be refreshed on the next bambi change instead of being re-discovered
- [ ] Remove the stale-fixture xfail marks in `tests/test_plotting_cartoon.py` (16 ids) and re-check the R2 marks on `tests/test_sample_posterior_predictive.py::test_sample_posterior_predictive`
- [ ] Consider a guard: fail loudly (or warn) when `sample_posterior_predictive` is handed a trace missing free RVs of the current model, so prior resampling cannot pass as posterior prediction

Details: `design/bambi-migration-fix-plan.md` (F6 checklist), `design/bambi-dev-upgrade-test-failures.md` (R10 status note).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the fixture loaders in tests/conftest.py, the model definitions for cav_model_cartoon and race_model_cartoon, and the fixtures under tests/fixtures/. Run the affected cartoon tests and tests/test_sample_posterior_predictive.py::test_sample_posterior_predictive to reproduce the failures. Done means regenerated fixtures, refreshed xfail decisions, a documented regeneration recipe, and a resolved decision on idata_race_cartoon.nc and the missing-RV guard.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.