mne-tools / mne-tools/mne-python

I/O roundtrip for annotations from a cropped raw induces a shift of raw.first_samp samples

Open
#11,731 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

If you crop a raw (with no measurement date), inducing a change in raw.first_samp, then save and reload the annotations attached, they are shifted by raw.first_samp. Let's consider a raw recording which has never been cropped and starts at raw.first_samp = 0, with 2 annotations at 2 and 4 seconds lasting 0.5 seconds:

from mne import Annotations, read_annotations
from mne.datasets import sample
from mne.io import read_raw_fif


fname = sample.data_path() / "MEG" / "sample" / "sample_audvis_raw.fif"
raw = read_raw_fif(fname, preload=False)
raw.pick_types(eeg=True)
raw.crop(0, 10, include_tmax=True)
raw.load_data()
raw.set_meas_date(None)  # remove measurement date since it impacts annotations
raw.set_annotations(None)
raw._cropped_samp = 0  # overwrite first_samp to 0 for this example
annots = Annotations(onset=[2, 4], duration=0.5, description="test", orig_time=None)
raw.set_annotations(annots)
raw.plot(n_channels=5)

Screenshot from 2023-06-13 13-26-16

Now let's crop the first second and set raw.first_samp to the 601 sample.

raw.crop(1, 8, include_tmax=True)
raw.plot(n_channels=5)

Screenshot from 2023-06-13 13-26-25

And finally, let's save the annotations to a separate file and load them back:

fname = "~/Downloads/test_annot.fif"
raw.annotations.save(fname, overwrite=True)
annots = read_annotations(fname)
raw.set_annotations(annots)  # overwrite existing annotations
raw.plot(n_channels=5)

Screenshot from 2023-06-13 13-26-32


By the end of the I/O roundtrip, the annotations are shifted by raw.first_samp, in this case 1 second. The "fix" is to edit the .onset attribute:

fname = "~/Downloads/test_annot.fif"
raw.annotations.save(fname, overwrite=True)
annots = read_annotations(fname)
annots.onset -= raw.first_samp / raw.info["sfreq"]
raw.set_annotations(annots)  # overwrite existing annotations
raw.plot(n_channels=5)

But Annotation.onset is not documented, I don't expect users to edit this attribute.


In the Notes of the Annotations class, there is this warning:

This means that when raw.info['meas_date'] is None, doing raw.set_annotations(raw.annotations) will not alter raw if and only if raw.first_samp == 0. When it’s non-zero, raw.set_annotations will assume that the “new” annotations refer to the original data (with first_samp==0), and will be re-referenced to the new time offset!

which seems related and is completely unclear to me.

raw.set_annotations(raw.annotations) will not alter raw [...]

Alter raw? How would that alter it if the raw.first_samp is different from 0?


Anyway, IMO, this I/O roundtrip should just work without users requiring to edit undocumented attributes of the re-loaded Annotations.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduced sequence using raw.crop, raw.set_annotations, Annotations.save, and read_annotations, then inspect how first_samp and meas_date are handled during annotation assignment and I/O. Verify the fix by rerunning the example and confirming that reloaded annotations retain their positions without manually changing onset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.