pyro-ppl / pyro-ppl/numpyro

GMM notebook example: MCMC/NUTS simulation is not reproducible

Open
#1,616 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted Tutorials/Examples
Dominant language
Python
Stars
2.8k
Forks
316
Avg merge
3d 9h
Merged PRs (30d)
27

Description

source: https://num.pyro.ai/en/stable/tutorials/gmm.html#MCMC
numpyro.__version__: 0.12.1
jax.__version__: 0.4.13

--

When running the collapsed NUTS to explore the full posterior, the obtained results did not match the presented ones.

from numpyro.infer import MCMC, NUTS


kernel = NUTS(model)
mcmc = MCMC(kernel, num_warmup=50, num_samples=250)
mcmc.run(random.PRNGKey(2), data)
mcmc.print_summary()
posterior_samples = mcmc.get_samples()

Obtained posterior density:
image

But, with longer num_warmup (to 150 or more), we get roughly the expected behaviour:

image

With more samples (~2500), the pattern is better:
image

I would like to clarify that in my previous attempts, I used the same values and parameters for reproduction. When I ran the code on Google Drive, the results matched the ones mentioned in the documentation. However, when I ran the code on my laptop, there was a significant difference. I can provide you with more details regarding this issue. Considering the specified random seeds and the simplicity of the example, I find the difference to be quite substantial.

I would like also to thank all the contributors for this library ! I am impressed and excited by the remarkable work done by its developers.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the GMM tutorial's MCMC section and run the reported code with NumPyro 0.12.1, JAX 0.4.13, and random.PRNGKey(2). Compare the posterior results across the laptop and Google Drive environments, then document or correct the example so the stated setup produces consistent results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.