HDFGroup / HDFGroup/HDF.PInvoke
[HELP] Wiki and demo code about read ultra large dataset
- Dominant language
- C#
- Stars
- 87
- Forks
- 31
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Many thanks for this great data format library.
I'm working on read a ultra large hdf5 dataset via HDF.PInvoke recently. Due to the reason of .NET array can not be allocated in size greater than 2GB, so if the data inside the hdf5 file its size is greater than 2GB, then the program crashed.
The dataset which its size less than 2GB could be read via the demo code in wiki [Cookbook : Datasets](https://github.com/HDFGroup/HDF.PInvoke/wiki/Cookbook-:-Datasets), and I guess i could read the dataset which its size greater than 2GB chunk by chunk via HDF.PInvoke, but after google search around it, i was not able to figure out how to implements such code correctly.
Here is the DEMO data in ultra large size that i currently working on: download the [P_1900 | Human Prostate](https://www.metabolomicsworkbench.org/data/DRCCMetadata.php?Mode=Study&StudyID=ST001857&StudyType=MS&ResultType=5) dataset from this repository.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.