openPMD / openPMD/openPMD-api

Series Open Performance

Open
#485 9 comments 0 reactions 0 assignees View on GitHub
good first issue help wanted internal
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

We should investigate and profile our performance of `Series` open (especially with `READ_ONLY` and `READ_WRITE`, but also with `CREATE` on truncation: #483).

Most crucial with file-based series, I have the subjective feeling they are slower than they need to be, already in serial mode.

We might need to clearly define also in this step, what we check and assume when opening a series (aka [contract](https://en.wikipedia.org/wiki/Design_by_contract)). The less assumptions the better, if you ask me. For example, I would not expect the data of each iteration nor it's names or layouts to be required to match. (Although some applications, such as our [openPMD-viewer](https://github.com/openPMD/openPMD-viewer/) GUI, will benefit if we later on provide an auxiliary `scan`-like list summary for records in all iterations.)

### Resources

Maybe some useful resources:

- [`scandir()`](https://geoff.greer.fm/2012/09/03/profiling-ag-writing-my-own-scandir/) in [`ag`](https://github.com/ggreer/the_silver_searcher)
- [benchmarking idea](https://geoff.greer.fm/2012/08/25/the-silver-searcher-benchmarking-revisions/)
- [`tracefile`](https://unix.stackexchange.com/questions/18844/list-the-files-accessed-by-a-program) (`strace` maybe expand to measure `time` spend; or even easier: add time log #462 to profile our `auxiliary::Filesystem` operations)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.