multiprocessing.pool.RemoteTraceback

General information:

• emcee version: 3.02
• platform: Centos7 cluster
• installation method (pip/conda/source/other?): pip

Problem description:

Expected behavior: as good as before

Actual behavior: stoped when running

What have you tried so far?:

Minimal example:

import emcee

# sample code goes here...
{
with Pool() as pool:
    sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, pool=pool,
                                                                    moves=[(emcee.moves.DEMove(), 1),]) 
    sampler.run_mcmc(starting_guesses, nsteps)
}


error:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/multiprocessing/pool.py", line    121, in worker
    result = (True, func(*args, **kwds))
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/multiprocessing/pool.py", line    44, in mapstar
    return list(map(*args))
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/site-packages/emcee/ensemble.py", line 545, in __call__
    return self.f(x, *self.args, **self.kwargs)
  File "DYB_simplified_STD-Pool.py", line 399, in log_posterior
    lp = log_likelihood(theta)
  File "DYB_simplified_STD-Pool.py", line 144, in log_likelihood
    print("chiSq : ", chiSq)
OSError: [Errno 116] Stale file handle
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "DYB_simplified_STD-Pool.py", line 446, in <module>
    sampler.run_mcmc(starting_guesses, nsteps)
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/site-packages/emcee/ensemble.py", line 384, in run_mcmc
    for results in self.sample(initial_state, iterations=nsteps, **kwargs):
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/site-packages/emcee/ensemble.py", line 343, in sample
    state, accepted = move.propose(model, state)
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/site-packages/emcee/moves/        red_blue.py", line 93, in propose
    new_log_probs, new_blobs = model.compute_log_prob_fn(q)
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/site-packages/emcee/ensemble.py", line 427, in compute_log_prob
    map_func(self.log_prob_fn, (p[i] for i in range(len(p))))
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/multiprocessing/pool.py", line    268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/public/data/yuebb/miniconda3/envs/ROOT/lib/python3.7/multiprocessing/pool.py", line    657, in get
    raise self._value
OSError: [Errno 116] Stale file handle

Start with the user’s minimal example and the traceback at DYB_simplified_STD-Pool.py line 144, where log_likelihood prints chiSq. Then inspect the referenced emcee/ensemble.py paths around lines 343, 384, and 427 to understand how the multiprocessing exception is propagated. Done means obtaining a reproducible example and establishing whether emcee or the cluster filesystem causes the stale file handle.