lnccbrown / lnccbrown/HSSM

[Migration] Make VI on the JAX compile backend trace bambi 0.20's symbolic __obs__ alloc

Open
#1,328 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
124
Forks
24
Avg merge
19h 32m
Merged PRs (30d)
60

Description

Part of #1305. Parent: #1306.

⚠️ **Not in [bambi#1002](https://github.com/bambinos/bambi/pull/1002)'s notes** — found by re-triage after #1310 unmasked the VI paths.

`HSSM.vi(...)` with `compile_kwargs={"mode": "JAX"}` fails while tracing the graph:

```
TypeError: Shapes must be 1D sequences of concrete values of integer type, got (JitTracer(int32[]),).
pytensor/link/jax/dispatch/tensor_basic.py:46: in alloc
... This concrete value was not available in Python because it depends on the value of the argument _obs_.
```

bambi 0.20's response term now allocates against a symbolic observation count (`__obs__`), and the JAX linker cannot make that shape static. Only the JAX-compiled VI rows fail: the pytensor rows pass, and so does `("jax", "advi", "reg_v")`, so the failure is shape- and method-dependent. MCMC on the same models is unaffected.

### Tasks
- [ ] Pin down which graph node introduces the symbolic `__obs__` alloc (bambi's response term vs. HSSM's distribution) and whether a `pm.Data`-style static shape is available
- [ ] Decide between fixing in HSSM (e.g. concrete length at `vi()` time) and reporting upstream to bambi/pytensor
- [ ] Remove the R11 xfail marks

### Impact
7 test ids across 2 files: `tests/integration/test_vi.py` (5), `tests/integration/test_missing_data_vi.py` (2).

Details: `design/bambi-dev-upgrade-test-failures.md` (R11), `design/bambi-migration-fix-plan.md` (F10).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the JAX-compiled VI failures in tests/integration/test_vi.py and tests/integration/test_missing_data_vi.py, then inspect the alloc traceback and the R11 notes in design/bambi-dev-upgrade-test-failures.md. Trace whether the symbolic __obs__ allocation comes from bambi's response term or HSSM, and check the migration fix plan in design/bambi-migration-fix-plan.md. Done means choosing an HSSM or upstream resolution and removing the seven R11 xfails.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.