H5Dread_chunk with partial reads
- Dominant language
- C
- Stars
- 988
- Forks
- 355
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 12
Description
I would love to be able to do a "partial read" of my raw chunk.
Some compression algorithm allow multi-level decompression, so doing a partial read, would give us information like:
1. what part of the chunk do we actually need to read to decode at a certain level.
I understand that this breaks some of the models in HDF5, but for certain very large dataset use cases, this kind of "multi-level reading" really does help.
# Describe the solution you'd like
Being able to provide a:
* offset within the chunk
* size of read
to read at the raw level.
# Describe alternatives you've considered
Saving the data in multiple resolutions.
That isn't fun now is it ;).
It just becomes hard to manage
# Additional context
I thought of doing this with the direct file pointer, but in certain cases, such as virtual dataset, it doesn't always exist.
Contributor guide
Assessment
This issue has not been assessed yet.