ValueError: set_state can only be used with legacy MT19937state instances.

General information:

• emcee version: 3.0.1
• platform: Scientific Linux 7.7, python 3.7.3
• installation method (pip/conda/source/other?): conda forge

Problem description:
When opening old chains saved as h5 files with python 2.7, I get this error: ValueError: set_state can only be used with legacy MT19937state instances.

What have you tried so far?:

I think this is python 2 vs 3 issue, since the variable state[0] in EnsembleSampler is a string. Saved with python 2 and read with python 3, it's a binary string, and thus needs to be converted to a proper string. This could be fixed by e.g. adding this line:

state[0]=state[0].decode()

before line 147 in ensemble.py.

Minimal example:

import emcee

filename = 'chain.h5' # already existing chain file, saved with python 2.7
backend = emcee.backends.HDFBackend(filename)
nwalkers = backend.shape[0]
ndim = backend.shape[1]
#following line results in error
sampler = emcee.EnsembleSampler(nwalkers, ndim, None, backend=backend)

Traceback

Traceback (most recent call last):
  File "example.py", line 8, in <module>
    sampler = emcee.EnsembleSampler(nwalkers, ndim, None, backend=backend)
  File "/anaconda3/lib/python3.7/site-packages/emcee/ensemble.py", line 147, in __init__
    self._random.set_state(state)
  File "mtrand.pyx", line 276, in numpy.random.mtrand.RandomState.set_state
ValueError: set_state can only be used with legacy MT19937state instances.

Inspect ensemble.py around line 147 and the HDFBackend state-loading path first. Reproduce the issue with the provided minimal example and an HDF5 chain saved under Python 2. Done means an old chain can initialize EnsembleSampler under Python 3 without the MT19937 state ValueError.