GMM notebook example: MCMC/NUTS simulation is not reproducible
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.8k
- Forks
- 316
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 27
Description
source: https://num.pyro.ai/en/stable/tutorials/gmm.html#MCMC
numpyro.__version__: 0.12.1
jax.__version__: 0.4.13
--
When running the collapsed NUTS to explore the full posterior, the obtained results did not match the presented ones.
from numpyro.infer import MCMC, NUTS
kernel = NUTS(model)
mcmc = MCMC(kernel, num_warmup=50, num_samples=250)
mcmc.run(random.PRNGKey(2), data)
mcmc.print_summary()
posterior_samples = mcmc.get_samples()
Obtained posterior density:
But, with longer num_warmup (to 150 or more), we get roughly the expected behaviour:
With more samples (~2500), the pattern is better:
I would like to clarify that in my previous attempts, I used the same values and parameters for reproduction. When I ran the code on Google Drive, the results matched the ones mentioned in the documentation. However, when I ran the code on my laptop, there was a significant difference. I can provide you with more details regarding this issue. Considering the specified random seeds and the simplicity of the example, I find the difference to be quite substantial.
I would like also to thank all the contributors for this library ! I am impressed and excited by the remarkable work done by its developers.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the GMM tutorial's MCMC section and run the reported code with NumPyro 0.12.1, JAX 0.4.13, and random.PRNGKey(2). Compare the posterior results across the laptop and Google Drive environments, then document or correct the example so the stated setup produces consistent results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100