HDFGroup / HDFGroup/h5pyd

hsload inefficient for zero-filled datasets

Open
#35 0 comments 0 reactions 1 assignee Claimed by @jreadey View on GitHub
Dominant language
Python
Stars
124
Forks
46
Avg merge
1h 24m
Merged PRs (30d)
6

Description

hsload isn't inspecting chunks prior to writing them to the server. This results in the server needlessly allocated chunks on the server and increased file size.

hsload should inspect each chunk and skip the write if the chunk is all zeros (or whatever the fill value is).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.