mne-tools / mne-tools/mne-python
Cropping data retains annotations that are outside the maximum range of data by 1 sample.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Describe the bug
Cropping an MNE Raw Array to a given time interval retains annotations that exist outside the time interval of the data. Specifically, annotations that occur up to 1 sample after the final sample in the data interval are retained.
I am not completely sure if this is a bug, or intended behavior.
It seems to be a bug, in that it seems to be unexpected, and as described below, writing a cropped file to BIDS format and reading it back in will generate a warning about the annotation being outside the range of the data.
However, the code itself seems to be explicitly capturing one sample beyond the range of the data, so perhaps it is intentional. Specifically, the issue seems to be related to the set_annotations method, where 1 sample is added to the end of the crop interval for purposes of selecting annotations:
https://github.com/mne-tools/mne-python/blob/main/mne/io/base.py#L657
Steps to reproduce
Here is a short example using a simulated dataset. In this example, annotations are placed at 0, 10, 20, and 30 seconds. The data is cropped to the region from 0 to 19.998 seconds (20 seconds not including tmax, at 500 Hz). In the cropped data, the annotation at 20 seconds, which is outside the data range, remains.
# MNE Cropping Example
import mne
import numpy as np
# generate a sample dataset from noise
n_channels = 2
sampling_freq = 500
length_seconds = 60
info = mne.create_info(n_channels, sfreq=sampling_freq, ch_types=['eeg', 'eeg'])
data = np.random.randn(n_channels, sampling_freq * length_seconds)
simulated_raw = mne.io.RawArray(data, info)
# insert annotations at 0, 10, 20, and 30 seconds
annotations = mne.Annotations(onset=[0, 10, 20, 30],
duration=[0, 0, 0, 0],
description=['test'] * 4)
simulated_raw.set_annotations(annotations)
# crop a copy of the data, setting 'include_tmax' to False, (up to but not including 20 seconds)
simulated_cropped = simulated_raw.copy().crop(tmin=0, tmax=20, include_tmax=False)
# final timestamp is 19.998, which is correct (data is at 500 Hz, we wanted up to but not including 20 seconds)
print('Final time is {}'.format(simulated_cropped.times[-1]))
# but the annotation that occurs at the 20 second point is retained
print(simulated_cropped.annotations[-1])
Expected results
I would expect that annotations outside the range of the data would be removed following the crop operation.
Actual results
As noted in the example above, an annotation that exists 1 sample outside of the crop time range will remain.
Additional information
Output of mne.sys_info():
Platform: macOS-10.15.7-x86_64-i386-64bit
Python: 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:20) [Clang 11.0.1 ]
Executable: /Users/danielroberts/anaconda/envs/mne/bin/python
CPU: i386: 8 cores
Memory: 16.0 GB
mne: 0.23.0
numpy: 1.20.2 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy: 1.6.2
matplotlib: 3.4.1 {backend=MacOSX}
sklearn: 0.24.1
numba: 0.53.1
nibabel: 3.2.1
nilearn: 0.7.1
dipy: 1.4.0
cupy: Not found
pandas: 1.2.3
mayavi: 4.7.2
pyvista: 0.29.0 {pyvistaqt=0.3.0, OpenGL 4.1 NVIDIA-14.0.32 355.11.11.10.10.143 via NVIDIA GeForce GT 750M OpenGL Engine}
vtk: 9.0.1
PyQt5: 5.12.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mne/io/base.py around the set_annotations logic linked in the issue, then reproduce the crop example with include_tmax=False. Verify how the final sample boundary is used when selecting annotations. Done means annotations outside the cropped data interval, including the one at 20 seconds in the example, are no longer retained and the behavior is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100