Chunk preallocation outside of dataset resize
- Dominant language
- C
- Stars
- 988
- Forks
- 355
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 12
Description
I use a chunked dataset for incremental I/O.
When I need to append data, I enlarge the dataset. Chunks are automatically allocated as needed.
For performance, since I approximately know how much data will be appended, I would like to pre-allocate chunks so that enlarging the dataset will be (hopefully) cheaper.
But :
- Would `H5Dset_extent(did, newExtent);` be really cheaper ? I assume so, but I can't be sure.
- Is it even possible to preallocate chunks without a hack involving "extend then shrink the dataset" ?
- Is "extend then shrink the dataset" a solution to force the dataset to allocate chunks ? I can't find documentation about sparse allocation (that would make it useless) and auto-vacuum (that would ruin the benefit of enlarge+shrink) that would tell how "extra" chunks are automatically handled.
Contributor guide
Assessment
This issue has not been assessed yet.