Use file based encoding to store BTD files with BP5
- Dominant language
- C++
- Stars
- 161
- Forks
- 59
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 7
Description
**Description:**
When running a BTD input, WarpX was hanging when using BP5, which uses PerformDataWrite(). And this function needs to be collective (and writing to the same ADIOS file).
The program hang when using file based encoding (e.g. one file per time step). I ran with two ranks. All was fine until at a certain point,
rank1 needs to write to step 1, the file is diags/diag2/openpmd_000001.bp
rank0 needs to write to step 0, the file is diags/diag2/openpmd_000002.bp
What happened in short is at step 2, both ranks on opening BP5 writer for file *00002, and BTD needs to write back to step 1 (file *00001) at rank 0 (but not rank 1).
When using PerformPut() instead of PerformDataWrite(), the run is successful because PerformPut waits till EndStep() to actually write out data, and EndStep() is collective.
When using group based encoding (e.g. all steps in one file), it works too as both ranks are writing to the same file.
**Work around:**
it looks like one option is to force BTD to be group based, or, we have an option to not call PerformDataWrite() in BP5 for BTD.
Contributor guide
Assessment
This issue has not been assessed yet.