openPMD / openPMD/openPMD-api

Read mode: Distinguish random access from streaming/linear access

Open
#1,234 0 comments 1 reaction 0 assignees View on GitHub
api: new backend: ADIOS2 feature request
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

**Is your feature request related to a problem? If so, please describe.**

Two angles to this:

1. Calling `Series("data.bp", Access::READ)` will implicitly parse the entire Series in random-access mode. If using `series.readIterations()` afterwards, this is unnecessary. Using a streaming read mode allows us to skip this, possibly simplifying the control flow in the ADIOS2 backend.
2. The upcoming BP5 engine in ADIOS2 will also distinguish between these two kinds of access modes. Random-accessing a series written via the streaming API will then only be possible by explicitly using the random access read mode.

**Describe the solution you'd like**
The existing `Access::READ_ONLY` keeps its semantics. a new mode `READ_LINEAR` or `READ_STREAMING` is added.

**Describe alternatives you've considered**
Deferred iteration parsing does similar things and will probably share some logic. It's not the same though:

1. A READ_LINEAR mode affects the moment when `BeginStep()` is first called during reading.
2. A READ_LINEAR mode will affect the mode that will be used to open the ADIOS2 engine.
3. Deferred iteration parsing does not change the behavior of the backend at all, it just does not descend into the single iterations while parsing initially.

**Additional context**
Further questions:

* What should be the behavior of this mode in file-based iteration encoding, group-based iteration encoding without steps, or other backends?
-> Probably similar to deferred iteration parsing
* As a perspective, we might merge deferred iteration parsing with this feature. But this will probably require an API addition for explicit random access (series.jumpToStep() or sth). For now, I would keep them separate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.