openPMD / openPMD/openPMD-api

ADIOS2: Non-MPI Parallel Reads (DASK)

Open
#1,500 0 comments 2 reactions 1 assignee Claimed by @ax3l View on GitHub
backend: ADIOS2 frontend: C++17 frontend: Python3
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

For DASK-style parallel reading, we would like to avoid the overhead of opening and parsing file meta data for chunked, parallel reads.

This strategy is implemented already in MPI-parallel backends, where one rank does the open and parsing and broadcasts to all other ranks involved.

Exposing this to non-MPI parallel reads is started by ADIOS2 via https://github.com/ornladios/ADIOS2/pull/3651 and we want to integrate it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.