openPMD / openPMD/openPMD-api

BP3 in ADIOS2: Reads Fail on Some Attributes

Open
#726 15 comments 0 reactions 3 assignees Claimed by @guj View on GitHub
affects latest release backend: ADIOS2 bug third party
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

**Describe the bug**
I seams there are problems reading a BP3 file from ADIOS1 with the ADIOS2 backend, with regards to reading the `/openPMD` base path string attribute: https://github.com/ComputationalRadiationPhysics/picongpu/issues/3223#issuecomment-612605933

**To Reproduce**

Create BP3 files with
```bash
export OPENPMD_BP_BACKEND="ADIOS1"
```

and then try to read generated files with ADIOS2:

```console
# ADIOS2 bpls
$ bpls -l -A ./samples/thetaMode_00200.bp
# ...
string /data/200/meshes/E/axisLabels attr = {"r", "z"}
string /data/200/meshes/E/dataOrder attr = "C"
string /data/200/meshes/E/geometry attr = "thetaMode"
string /data/200/meshes/E/geometryParameters attr = "m=4;imag=+"
double /data/200/meshes/E/gridGlobalOffset attr = {0, 0}
double /data/200/meshes/E/gridSpacing attr = {1, 1}
# ...
string /openPMD attr = "1.1.0"
uint32_t /openPMDextension attr = 0
string /software attr = "openPMD-api"
string /softwareVersion attr = "0.12.0-dev"
```
So far so good, but `/data/200/meshes/E/gridUnitSI` is missing (the unit test that writes this data [even sets it explicitly](https://github.com/openPMD/openPMD-api/blob/0.11.1-alpha/test/SerialIOTest.cpp#L946).)

If I write the same data set with ADIOS2, the `/data/200/meshes/E/gridUnitSI` is present.

Python read
```console
export OPENPMD_BP_BACKEND="ADIOS2"
export PYTHONPATH=$PWD/lib/python3.7/site-packages/:$PYTHONPATH
```
then
```python
import openpmd_api as io

s = io.Series("samples/thetaMode_%T.bp", io.Access_Type.read_only)
```
```
Warning: ADIOS2 is not configured in the JSON configuration. Running with default settings.
[ADIOS2] Warning: Attribute with name /data/200/meshes/E/gridUnitSI has no type in backend.
[ADIOS2] Warning: Attribute with name /data/200/meshes/E/gridUnitSI has no type in backend.
[~ADIOS2IOHandler] An error occurred: [ADIOS2] Requested attribute (/data/200/meshes/E/gridUnitSI) not found in backend.
Traceback (most recent call last):
File "", line 1, in
RuntimeError: [ADIOS2] Requested attribute (/data/200/meshes/E/gridUnitSI) not found in backend.
```

**2nd Bug:** With PIConGPU ADIOS1 files ... (need to create new ones)
```python
import openpmd_api as io

s = io.Series("simData_%T.bp", io.Access_Type.read_only)
```
```
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
terminate called after throwing an instance of 'std::runtime_error'
what(): [ADIOS2] Requested attribute (/openPMD) not found in backend.
```

**Expected behavior**
Should be readible via the ADIOS2 BP3 engine.

**Software Environment**
- version of openPMD-api: 0.11.0 and `dev` as of d7a16a171873b435aca24b1a6a09a3dab46dd812
- installed openPMD-api via: from source (and user report below)
- operating system: Ubuntu 18.04
- machine: local laptop
- name and version of Python implementation: CPython 3.7.7
- version of HDF5: 1.10.6
- version of ADIOS1: 1.13.1
- version of ADIOS2: 2.5.0
- name and version of MPI: MPICH 3.3.2

**Additional context**
First seen in https://github.com/ComputationalRadiationPhysics/picongpu/issues/3223#issuecomment-612605933
with PIConGPU data:
```
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
terminate called after throwing an instance of 'std::runtime_error'
what(): [ADIOS2] Requested attribute (/openPMD) not found in backend.
```

P.S.: should we really print a warning
```
Warning: ADIOS2 is not configured in the JSON configuration. Running with default settings.
```
?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.