dfm / dfm/emcee

different results when using multiprocessing Pool() (sometimes)

Open
#351 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
440
PR merge metrics
No merged PRs in 30d

Description

General information:

  • emcee version: 2.2.1
  • platform: Windows
  • installation method (pip/conda/source/other?): pip

Problem description:
I was doing some sanity checks with the line.py example and run into different results whether or not I used multiprocessing Pool() when initializing EnsembleSampler (or PTSampler for that matter). Under other circumstances the results are in fact the same.

Minimal example:

These are the modification to line.py I made:

from multiprocessing import Pool
### same code
# widen the parameter ranges
def lnprior(theta):
    m, b, lnf = theta
    if -50.0 < m < 50 and -50.0 < b < 50 and -50.0 < lnf < 50:
        return 0.0
    return -np.inf
### same code
# less optimal initial positions and sample more times
pos = [result["x"] + 5*np.random.randn(ndim) for i in range(nwalkers)]
sampler = emcee.EnsembleSampler(nwalkers, ndim, lnprob, args=(x, y, yerr), pool = None) # or pool = Pool()
sampler.run_mcmc(pos, 1500, rstate0=np.random.get_state())

A rough example based on the code above.
image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the line.py example and reproduce the comparison between pool=None and multiprocessing Pool() on Windows using the reported emcee 2.2.1 setup. Read the EnsembleSampler and PTSampler initialization and run_mcmc paths, focusing on how the pool and rstate0 are handled. Done means the discrepancy is consistently reproduced and its cause is identified or the seeded runs produce equivalent results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.