HDFGroup / HDFGroup/h5pyd

Variable length array

Open
#85 2 comments 0 reactions 1 assignee Claimed by @jreadey View on GitHub
Dominant language
Python
Stars
124
Forks
46
Avg merge
1h 24m
Merged PRs (30d)
6

Description

I'm creating a backend to use transparently pytables, h5py and h5pyd. I ran my test suite on h5pyd and am confronted with the issue of the variable length array. In h5py, they manage to do it using a "special" dtype: `h5py.string_dtype` or `h5py.vlen_dtype` . After some digging in I found in h5pyd, the function `special_dtype` where the docstring seems promising:
```
vlen = basetype
Base type for HDF5 variable-length datatype. This can be Python
str type or instance of np.dtype.
Example: special_dtype( vlen=str )
```
however after trying it out it seems its working only in the case where `vlen=str` and not any numpy dtype. Using a special dtype of np.uint32s I could create a dataset but when trying to access a given element I got this traceback:
`File "", line 1, in
dset[0]
File "C:\Miniconda3\envs\pymodaq_dev\lib\site-packages\h5pyd\_hl\dataset.py", line 802, in __getitem__
arr1d = bytesToArray(rsp, mtype, page_mshape)
File "C:\Miniconda3\envs\pymodaq_dev\lib\site-packages\h5pyd\_hl\base.py", line 503, in bytesToArray
offset = readElement(data, offset, arr, index, dt)
File "C:\Miniconda3\envs\pymodaq_dev\lib\site-packages\h5pyd\_hl\base.py", line 467, in readElement
arr[index] = vlen(0)
TypeError: 'numpy.dtype' object is not callable`

Then a bit further in the code I found :

> def check_dtype(**kwds):
> """ Check a dtype for h5py special type "hint" information. Only one
> keyword may be given.
>
> vlen = dtype
> If the dtype represents an HDF5 vlen, returns the Python base class.
> Currently only builting string vlens (str) are supported. Returns
> None if the dtype does not represent an HDF5 vlen.

So the question is: is it or will it be possible to use any numpy dtype for variable length arrays in h5pyd?

Thx

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.