emcee and MPI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 440
- PR merge metrics
- No merged PRs in 30d
Description
General information:
- emcee version: 3.0.2
- platform:
- installation method (pip/conda/source/other?): pip
I am running the MPI example (taken from https://emcee.readthedocs.io/en/stable/tutorials/parallel/#mpi) on a cluster using "--ntasks 10". The execution seem to reach a completion when using 32 walkers, however when I increase the number of walkers above a certain number (e.g. 200) the script appears to be freezing (e.g. the progress bar is stuck at zero) and the job will use all the available execution time but will not Fail.
Do I run in to this problem because I am not using enough tasks? Is there a rule of thumb as to how many tasks I need to be using on a cluster for a given number of nwalkers?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the MPI example in the emcee parallel tutorial and reproduce the reported behavior with 10 tasks, comparing 32 walkers with about 200 walkers. Check the execution and communication behavior around the progress bar, then document whether the task-to-walker relationship explains the stall and what configuration guidance follows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100