G-Node / G-Node/nixpy

Variable or fixed length string data

Open
#238 3 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
22
Forks
29
PR merge metrics
No merged PRs in 30d

Description

I've been looking into supporting string type data in DataArrays (see old issue #82).
I think this is doable but there's one decision to consider and I would like some feedback.

We can support variable length or fixed length strings as data in DataArrays, or maybe both in some special way, and here are the arguments I can think of for each.
## Variable length
- Obviously this is more flexible. Once a DataArray is created it can be edited freely.
- We already use variable length strings in [Property](https://github.com/G-Node/nixpy/blob/2d33d77fe9ccca2ea7065918c32e096026e2444f/nixio/pycore/property.py#L114) for string type metadata and [Dimension](https://github.com/G-Node/nixpy/blob/86ff40667a423c3640697be4cabe929dbd88d3b2/nixio/pycore/dimensions.py#L269) for labels.
- **Potential issue**: The [special dtype](https://github.com/G-Node/nixpy/blob/10e48a3b83bffed7e68d7a5fb5352b4d8e69d74a/nixio/pycore/util/util.py#L34) we create to support variable length strings is simply of type `object` (`np.dtype('O')`). This isn't an issue now since no other data type we use matches this type. We could just assume that any data we read back that's of type `O` is a variable length string.

## Fixed length
- We could restrict strings in DataArray to be fixed length, the same way any other type of DataArray can be reshaped but not resized. A string in this instance would be useful for storing vectors of character data and not necessarily text.
- The stored data type would not be lost on read. We would get back a `np.dtype('|Sn')`, where `n` is the length of the string.

----------
I'd love to hear everyone's thoughts on this.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.