Error concatenating epochs, when one epochs object contains duplicate epochs

Description of the problem

When concatenating epochs, in each epochs object the single epochs need a different value in the .selection attribute. Suppose I have duplicate epochs (therefore duplicate .selection values) within one of the two epochs objects. In that case, it does not add an "offset" to the duplicate values within one epochs object, as typically done during concatenation between two epoch objects. Instead, it throws a ValueError. Therefore, you can not create an epochs object by subsampling with a list as long there are duplicates in the list, and if you want to concatenate these subsampled epochs with other epochs.

Workaround: When subsampling epochs with a list containing duplicates, one can introduce another concatenation step that alters the .selection values, before doing the intended concatenation with the other epochs object.

Steps to reproduce

import mne

# Load the example dataset
data_path = mne.datasets.sample.data_path()
raw = mne.io.read_raw_fif(f'{data_path}/MEG/sample/sample_audvis_raw.fif', preload=True)

# Define event IDs and tmin/tmax for epochs
event_id = {'Auditory/Left': 1, 'Auditory/Right': 2}
tmin, tmax = -0.2, 0.5  # Define epoch time window

# Define events based on triggers in the raw data
events = mne.find_events(raw, stim_channel='STI 014')

# Create epochs from the raw data
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, baseline=(None, 0), preload=True)

# Print some information about the loaded epochs
print(epochs)

# concatenate two copies of the same epochs
list_without_duplicates = [0,1,2,3,4]
epochs1 = epochs['Auditory/Left'].copy()[list_without_duplicates]
epochs2 = epochs['Auditory/Left'].copy()[list_without_duplicates]
list_with_duplicates = [0,1,1,2,1]
epochs3 = epochs['Auditory/Left'].copy()[list_with_duplicates]

# this works fine
epochs_concat_fine = mne.concatenate_epochs([epochs1, epochs2])
print(epochs1.selection)
print(epochs2.selection)
print(epochs_concat_fine.selection)

# this fails, due to duplicates of some epochs within epochs3
epochs_concat_fail = mne.concatenate_epochs([epochs1, epochs3])
print(epochs1.selection)
print(epochs3.selection)

# workaround
epochs3_base = [epochs['Auditory/Left'].copy()[i] for i in list_with_duplicates]
epochs3_workaround = mne.concatenate_epochs(epochs3_base)
epochs_concat_workaround = mne.concatenate_epochs([epochs1, epochs3_workaround])
print(epochs1.selection)
print(epochs3_workaround.selection)
print(epochs_concat_workaround.selection)

Link to data

No response

Expected results

No ValueError. epochs_concat_fail should contain 10 epochs. Duplicate epochs should have different .selection values.

Actual results

ValueError Traceback (most recent call last)
Cell In[15], line 5
2 print(f'epochs_concat_fine: {epochs_concat_fine}')
3 # this fails, due to duplicates of some epochs
----> 5 epochs_concat_fail = mne.concatenate_epochs([epochs1, epochs3])
6 print(f'epochs_concat_fail: {epochs_concat_fail}')

File :12, in concatenate_epochs(epochs_list, add_offset, on_mismatch, verbose)

File ~/conda-envs/prawn/lib/python3.11/site-packages/mne/epochs.py:4208, in concatenate_epochs(epochs_list, add_offset, on_mismatch, verbose)
4189 (
4190 info,
4191 data,
(...)
4205 on_mismatch=on_mismatch,
4206 )
4207 selection = np.where([len(d) == 0 for d in drop_log])[0]
-> 4208 out = EpochsArray(
4209 data=data,
4210 info=info,
4211 events=events,
4212 event_id=event_id,
4213 tmin=tmin,
4214 baseline=baseline,
...
523 )
524 self.selection = selection
525 if drop_log is None:

ValueError: selection must be shape (10,) got shape (8,)

Additional information

Platform Linux-5.14.21-150400.24.81-default-x86_64-with-glibc2.31
Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0]
Executable - hidden -
CPU x86_64 (144 cores)
Memory 503.1 GB

Core
├☑ mne 1.5.0
├☑ numpy 1.25.2 (OpenBLAS 0.3.23 with 128 threads)
├☑ scipy 1.11.2
├☑ matplotlib 3.7.2 (backend=module://matplotlib_inline.backend_inline)
├☑ pooch 1.7.0
└☑ jinja2 3.1.2

Numerical (optional)
├☑ sklearn 1.3.0
├☑ pandas 2.0.3
└☐ unavailable numba, nibabel, nilearn, dipy, openmeeg, cupy

Visualization (optional)
└☐ unavailable pyvista, pyvistaqt, ipyvtklink, vtk, qtpy, ipympl, pyqtgraph, mne-qt-browser

Ecosystem (optional)
└☐ unavailable mne-bids, mne-nirs, mne-features, mne-connectivity, mne-icalabel, mne-bids-pipeline

Start at mne/epochs.py, especially concatenate_epochs and the EpochsArray construction shown in the traceback. Reproduce the failure with duplicate selections from the issue, then inspect how concatenation builds events, drop_log, and selection. Done means concatenating epochs1 and epochs3 returns 10 epochs with distinct selection values and a regression test covers the case.