mne-tools / mne-tools/mne-python

DOC: Epoch loading shifts index

Open
#11,373 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DOC EASY
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

One epoch in my dataset was automatically removed due to it being too close to the start of the experiment (making it impossible to apply a baseline with negative times). Due to this, the indices displayed in epochs.plot() and the bad epochs indices that are printed after going through the epochs are shifted by one index (i.e. removing the first and second epochs removes [1, 2] not [0, 1]). So far so good. However, epochs.drop() starts with index 0 as if the automatically dropped epoch had never existed. As a result, in the case of the example, not epochs 1 and 2 get dropped, but 2 and 3.

Possibly related to #7260

Steps to reproduce
import mne

# Load the sample dataset.
sample_data_folder = mne.datasets.sample.data_path()
sample_data_raw_file = os.path.join(sample_data_folder, 'MEG', 'sample',
                                    'sample_audvis_raw.fif')

raw = mne.io.read_raw_fif(sample_data_raw_file).crop(tmax=60)
events = mne.find_events(raw, stim_channel='STI 014')

# Set the first event to time index 0, such that it'll be too close to the start of the data.
events[0, 0] = 0

# Now try to extract the epoch with a baseline before it.
epochs = mne.Epochs(raw, events, tmin=-1, tmax=0.7, baseline=(-1, 0))

# Plot the epochs, the first epoch will be automatically dropped.
epochs.plot()

# Now if we click on the epochs that we want to drop in the plot,
# the dropping will work as expected. But if we take the list of
# dropped epochs, recreate the epochs object and use those indices
# to drop the epochs automatically, the dropped epochs will be shifted.
# Let's try just that. Imagine we just dropped epoch 1 in the plot.
# We get the following array:
dropped_epochs = [1]

# Now we drop this epochs and plot again.
epochs.drop(dropped_epochs)
epochs.plot()

# In the plot you should see that not epoch 1, but epoch 2 got dropped.
Link to data

No response

Expected results

Either epoch 1 is dropped when giving index 1 to epochs.drop() or epochs.plot() is modified such that the index starts at 0 when epochs are automatically dropped.

Actual results

The indices in epochs.plot() do not correspond to the epoch indices in epoch.drop() after epochs are dropped automatically.

Additional information

Output exceeds the size limit. Open the full output data in a text editor
Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Python: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
Executable: /home/nathan/Workspace/NE2/Cybathlon/venv/bin/python
CPU: x86_64: 4 cores
Memory: 7.7 GB

mne: 1.2.3
numpy: 1.23.0 {OpenBLAS 0.3.20 with 4 threads}
scipy: 1.8.1
matplotlib: 3.5.2 {backend=module://matplotlib_inline.backend_inline}

sklearn: 1.1.1
numba: 0.56.4
nibabel: Not found
nilearn: Not found
dipy: Not found
openmeeg: Not found
cupy: Not found
pandas: 1.4.3
pyvista: Not found
pyvistaqt: Not found
ipyvtklink: Not found
vtk: Not found
qtpy: Not found
ipympl: Not found
...
mne_features: 0.2.1
mne_qt_browser: Not found
mne_connectivity: Not found
mne_icalabel: Not found

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Epochs behavior exercised by the reproduction, especially epochs.plot() and epochs.drop() after an automatically rejected epoch. Reproduce the index mismatch using the sample dataset and the provided events and baseline settings. Done means the index selected in the plot corresponds to the same epoch removed by epochs.drop(), with the expected behavior covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.