Overhead with variable-based `bp5` series
- Dominant language
- C++
- Stars
- 161
- Forks
- 59
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 7
Description
**Describe the bug**
Performance kill when opening/closing a variable-based `bp5` series with many iterations.
**To Reproduce**
Standalone reproducer in Python:
```python
import openpmd_api as io
from tqdm import tqdm
adios2_cfg = r'''
{
"iteration_encoding": "variable_based",
"adios2": {
"engine": {
"type": "bp5",
"parameters": {
"StatsLevel": "0"
}
}
}
}
'''
OUTPUT_PATH = "myseries.bp5"
series = io.Series(OUTPUT_PATH, io.Access_Type.create_linear, adios2_cfg)
N = 1000000
for i in tqdm(range(N)):
it = series.snapshots()[i]
it.close()
```
**Expected behavior**
I expect much less overhead just to open/close the iterations, and not to change in time.
**Software Environment**
- version of openPMD-api: 0.17.0
- installed openPMD-api via: conda-forge
- operating system:
- machine: Perlmutter at NERSC - login nodes
- name and version of Python implementation: CPython 3.11.14
- version of ADIOS2: 2.11.0
- name and version of MPI: N/A
**Additional context**
The overhead builds up as the loop moves through the iterations.
Starts around ~1k iterations/s and goes down to ~200 iterations/s after ~100k iterations.
Contributor guide
Assessment
This issue has not been assessed yet.