dfm / dfm/emcee

a small issue with the global_variable+pool approach when using Python 3.8 in macOS

Open
#335 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
440
PR merge metrics
No merged PRs in 30d

Description

I just upgraded to Python 3.8 on a Mac and found that my previously working code (under 3.7) broke down, as well as the "Multiprocessing+global-variable" method presented at the end of this tutorial. It turns out Python 3.8 in macOS switches to the spawn start method in multiprocessing as default, so the resources of the parent process may not be inherited by the child process. Then the "global" variables become inaccessible.

A short workaround may be changing

from multiprocessing import Pool

into

import multiprocessing as mp
Pool = mp.get_context('fork').Pool

in the tutorial example. This is just to make sure that the fork method is always used.

This issue is not an emcee problem at all, and will only happen for certain Python versions + platforms. But the tutorial could be slightly tweaked for this.

btw, thank you for writing the excellent documentation for the package.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the parallel tutorial linked in the issue, focusing on the global-variable and Pool example and its multiprocessing imports. Check the Python 3.8 macOS spawn behavior described there, then update the example so its documented workaround consistently uses fork; the tutorial should explain how the example remains usable on that platform.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.