G-Node / G-Node/nix

Tag data retrieval specification

Open
#818 2 comments 0 reactions 0 assignees View on GitHub
discussion
Dominant language
C++
Stars
72
Forks
36
PR merge metrics
No merged PRs in 30d

Description

We need to clarify and specify unambiguously how data retrieval works for Tag and MultiTag referenced data. I'm going to ignore Untagged and Indexed features for now, because I think these are simple and clear. So this refers to referenced data and the Tagged features which behave equivalently.

In all the examples below, I'll be specifying positions, extents, and timestamps. Assume for simplicity that timestamps is a range dimension on the referenced data (or a data array with alias range dimension) with the same unit as the positions and extents.

Currently there's a bug where data is returned even when the tag region should be empty. Consider the following example:
```
positions = [0]
extents = [10]
timestamps = [22, 23, 31]
```
The first tag index should return an empty DataView but instead returns `[22]`. This seems to be because we first check if there's an index/tick that's **greater or equal** than the positions, to set it as a start index. This is fine when there's at least one value to return (the first value returned should be at an index that's `>=` to the start position), but should not apply when the end position (`position + extent`) is less than the first available index or tick.

I also want to clarify that at some point we decided that the end position should be _inclusive_. Is this correct? For example:

```
positions = [10]
extents = [10]
timestamps = [10, 12, 19, 20, 23]
```

- **Inclusive** returns: `[10, 12, 19, 20]`
- **Non-inclusive** returns: `[10, 12, 19]`

Currently we decided on inclusive but I'm wondering if we would like to revisit this. I think the inclusive slicing is a little unexpected, at least in certain cases. For instance:

```
positions = [0, 10, 20, 30]
extents = [10, 10, 10, 10]
timestamps = [0, 1, 2, 3, 4, ..., 39]
```

I think many would expect that retrieving the slices defined by each tag would have no overlap.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Tag and MultiTag referenced-data retrieval paths and reproduce the empty-region example from the issue. Compare the current behavior with the inclusive and non-inclusive boundary cases. Done requires an unambiguous specification and agreed behavior for empty regions and end-position handling; the payload names no files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.