google-deepmind / google-deepmind/torch-hdf5

Support for partial writing

Open
#45 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Lua
Stars
241
Forks
125
PR merge metrics
No merged PRs in 30d

Description

In the docs there is a section on partial reading:

``` lua
local myFile = hdf5.open('/path/to/read.h5','r')
-- Specify the range for each dimension of the dataset.
local data = f:read('/path/to/data'):partial({start1, end1}, {start2, end2})
myFile:close()
```

it would be great if there was also support for partial writing, when making datasets that don't fit in RAM. For instance, ideally something along the lines

``` lua
local data = f:write('/path/to/data', data):partial({start1, end1}, {start2, end2})
```

I don't believe this is currently supported, or at least it's not documented in the docs.

Contributor guide

Open the contributing guide

Research direction

Start with the documentation section and existing partial-reading API shown in the issue, then inspect the repository for the corresponding write path. Determine whether partial writing already exists but is undocumented or requires new support; done means the intended partial-write behavior is implemented and documented, with coverage for datasets that are written in ranges.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.