stan-dev / stan-dev/math

Use Stan's own MPI communicator to avoid communicator interference

Open
#1,165 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
839
Forks
220
Avg merge
2d 4h
Merged PRs (30d)
14

Description

Description

This is a following up, or more specific issue spawned from https://github.com/stan-dev/math/issues/939. Namely, currently Stan's MPI implementation uses a predefined global communicator MPI_COMM_WORLD, this is against the recommended use of the communicators, see

https://www.mcs.anl.gov/research/projects/mpi/tutorial/gropp/node116.html
https://www.open-mpi.org/doc/current/man3/MPI_Comm_dup.3.php#toc8

Specifically, any communication, even if using a local communicator(as an application should be), will be polluted by map_rect 's communicator, whether that local communicator is within or out of Stan. This causes unpredictable behavior, therefore is a design flaw.

Proposed short-term solution

Replace construction boost::mpi::communicator world calls with
boost::mpi::communicator world(MPI_COMM_WORLD, comm_duplicate)

Proposed long-term solution

Address https://github.com/stan-dev/math/issues/939.

Expected Output

With current tests, same. But need add more tests with new communicators.

Current Version:

v2.18.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the Boost.MPI communicator constructions used by Stan's MPI implementation, especially the world instances described in the issue, and review the existing MPI tests. Verify the communicator handling against the current tests, then add coverage using separate communicators and confirm that communication remains isolated.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.