ADIOS2: Non-MPI Parallel Reads (DASK)
Open
backend: ADIOS2
frontend: C++17
frontend: Python3
- Dominant language
- C++
- Stars
- 161
- Forks
- 59
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 7
Description
For DASK-style parallel reading, we would like to avoid the overhead of opening and parsing file meta data for chunked, parallel reads.
This strategy is implemented already in MPI-parallel backends, where one rank does the open and parsing and broadcasts to all other ranks involved.
Exposing this to non-MPI parallel reads is started by ADIOS2 via https://github.com/ornladios/ADIOS2/pull/3651 and we want to integrate it.
Contributor guide
Assessment
This issue has not been assessed yet.