Multiprocessing the Python module 'emcee', but not all CPU cores on the machine are being used

Hello,

I am parallelizing emcee using multiprocessing module as stated in the emcee document. However, htop shows that the program keeps using a limited number of cores (26-27). The computer I am running my code on has 80 cores and I would like to make use of all of them to speed up the code. Could you please help me with this? I did not find any solution in the document. This only happens with emcee and when I use multiprocessing module in other programs I see with htop that all 80 cores are being used.

This is my code:

with Pool() as pool:

        sampler = emcee.EnsembleSampler(nwalkers, npars, logpfunc, pool=pool)

        start = []
        if len(sys.argv) > 1:
            print('using last spread of %s to initialize walkers'%sys.argv[1])
            startfile = h5py.File('%s'%sys.argv[1], 'r')

            for i in range(nwalkers):
                tmp = np.zeros(npars)
                for nn in range(npars):
                    tmp[nn] = startfile[pars[nn]['name']][i, -1]
                start.append(tmp)
            startfile.close()

        else:
            for i in range(nwalkers):
                tmp = np.zeros(npars)
                for j in range(npars):
                    a, b = (bounds[j][0] - pars[j]['guess'])/pars[j]['spread'], (bounds[j][1] - pars[j]['guess'])/pars[j]['spread']
                    p0 = truncnorm.rvs(a, b, size=1)*pars[j]['spread'] + pars[j]['guess']
                    tmp[j] = p0

                start.append(tmp)

        print('Sampling')

        sampler.run_mcmc(start, nstep, progress=True)

Start with the emcee documentation for multiprocessing, then inspect the EnsembleSampler pool parameter and the sampler.run_mcmc call shown in the report. Reproduce the limited-core behavior and determine whether it is caused by emcee's sampling behavior or the multiprocessing setup; done means the cause and a supported resolution are documented.