mne-tools / mne-tools/mne-python

Annotation timing info loss due to datetime.timedelta precision limit

Open
#12,327 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

Sub-microsecond timing data is lost when pushed through a datetime.timedelta object step, for example in mne.annotations.crop. As raw.set_annotations calls crop internally, this can lead to misalignment of the created annotations and raw.times, with the annotations covering slightly different frames than expected.

This will only affect cases where 1/sfreq has sub-microsecond precision, but 512Hz is common in my experience, and 1/512 = 0.001953125s.

Steps to reproduce
from datetime import datetime, timezone

import numpy as np
from mne import Annotations, create_info
from mne.io import RawArray

info = create_info(5, 512, "eeg")
data = np.random.randn(5, 10000)
raw = RawArray(data, info)
print(raw.times[1])
# 0.001953125
annotation = Annotations(onset=raw.times[1], duration=1, description="test")
print(annotation[0])
# OrderedDict([('onset', 0.001953125), ('duration', 1.0), ('description', 'test'), ('orig_time', None)])
raw.set_annotations(annotation)
print(raw.annotations[0])
# OrderedDict([('onset', 0.001953), ('duration', 1.0), ('description', 'test'), ('orig_time', None)])
print(raw.time_as_index(raw.annotations[0]['onset']))
# array([0])
Link to data

No response

Expected results

The annotation details shouldn't be lost when attached to the raw object. The correct original range of frames should be as precisely obtainable from the timing data as before attached to the raw object.

annotation[0]['onset'] and raw.annotations[0]['onset'] should be equal at 0.001953125.
raw.time_as_index(raw.annotations[0]['onset']) should return array([1]), corresponding to how the annotation was created.

Actual results

annotation[0]['onset'] is 0.001953125 while raw.annotations[0]['onset'] is 0.001953.
raw.time_as_index(raw.annotations[0]['onset']) returns array([0])

Additional information

Platform Linux-3.10.0-1160.102.1.el7.x86_64-x86_64-with-glibc2.17
Python 3.11.5 (main, Sep 5 2023, 13:23:33) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
Executable /isilon/LFMI/VMdrive/Jonathan/PriorWeightingMooney/Visual/ECoG/Code/.venv/bin/python
CPU x86_64 (20 cores)
Memory 96.2 GB

Core
├☑ mne 1.5.1
├☑ numpy 1.26.2 (OpenBLAS 0.3.23.dev with 20 threads)
├☑ scipy 1.11.4
├☑ matplotlib 3.8.2 (backend=module://matplotlib_inline.backend_inline)
├☑ pooch 1.8.0
└☑ jinja2 3.1.2

Numerical (optional)
├☑ sklearn 1.3.2
├☑ nibabel 5.1.0
├☑ nilearn 0.10.2
├☑ pandas 2.1.3
└☐ unavailable numba, dipy, openmeeg, cupy

Visualization (optional)
├☑ pyvista 0.42.3 (OpenGL unavailable)
├☑ pyvistaqt 0.11.0
├☑ vtk 9.3.0
├☑ ipympl 0.9.3
├☑ ipywidgets 7.8.1
└☐ unavailable qtpy, pyqtgraph, mne-qt-browser, trame_client, trame_server, trame_vtk, trame_vuetify

Ecosystem (optional)
└☐ unavailable mne-bids, mne-nirs, mne-features, mne-connectivity, mne-icalabel, mne-bids-pipeline

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at mne/annotations.py around Annotations.crop and trace how raw.set_annotations passes timing values through it. Run the reproduction in the issue, then verify that attached annotation onset retains 0.001953125 and raw.time_as_index returns array([1]) without losing sub-microsecond precision.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.