Better documentation of behaviour with irregular source chunks
- Dominant language
- Python
- Stars
- 170
- Forks
- 15
- Avg merge
- 18h 27m
- Merged PRs (30d)
- 1
Description
Hiya!
In some source datasets the original chunks have irregular sizes. For example one netcdf file per year, where some years are leap years hence shorter than others.
Although the data model would seem to support this in principle, I couldn't get Rechunk or ConsolidateChunks to work without regular source chunk sizes.
Is this a fundamental limitation or something that could be in scope to address? And do you have any recommendations for getting data like this into regular chunks? In the leap year case, first splitting the source chunks into one chunk per day using SplitChunks could be an option. Although in general this would require splitting into chunks of `gcd(*all_possible_source_chunk_sizes)` which could be too small.
Contributor guide
Assessment
This issue has not been assessed yet.