gridap / gridap/GridapSolvers.jl

Optimizing communications when redistributing models

Open
#32 0 comments 1 reaction 2 assignees Claimed by @amartinhuertas View on GitHub
enhancement
Dominant language
Julia
Stars
43
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Current implementation:
- When creating an N-processor sub-communicator, we are always picking `N` the processors with lowest id (i.e [1,...,N]).
- When redistributing a mesh using P4est, we are partitioning cells using a so-called Z-space-filling curve. This assigns cell ids to processors in a consecutive way using the numbering given by the Z-space-filling curve.

This leads to certain situations with obvious sub-optimal communications. For instance, consider a cartesian mesh distributed between 2 processors that gets redistributed to 4 processors:

![image](https://github.com/gridap/GridapSolvers.jl/assets/37735424/479434ca-7f8f-4c6f-affe-bcfae14ed624)

The current implementation returns a communication pattern `1 -> [2], 2 -> [3,4]` which involves moving 75% of the mesh every time we want to redistribute information. The optimal communication pattern would be `1 -> [3], 2 -> [4]`, which would only communicate 50% of the mesh every time.

Changing the behavior of the z-space-filling curve is quite involved, so to solve this issue we might want to have a look a how the sub-communicators are created. For instance taking `[1,3]` as the coarse sub-communicator would potentially yield a better communication pattern for the example above.

To-Do:
- [ ] Use profilers to identify if this problem is indeed a bottleneck. Compare performance with topology-optimized communicators.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.