mne-tools / mne-tools/mne-python

In `crop(tmin)` versus `get_data(tmin)`, `tmin` has a different meaning

Open
#13,634 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

Hi,

Thanks again for the work MNE, I use it and recommend it all the time.

I was bitten recently by an off-by-one error that I ended up tracing back to the fact that epoch.crop(tmin) and epoch.get_data(tmin) interpret tmin differently. The easiest is probably to show a minimal working example below.

Thanks,

Steps to reproduce

Running the following code:

import mne
import numpy as np

# Create a datastet w/ single sensor, 100Hz, ramping value matching time
data = np.linspace(-3.5, 1, 451).reshape((1,1,451))
epochs = mne.EpochsArray(data, mne.create_info(['test'], 100), tmin=-3.5, verbose=False)

t = 0.77
idx_method = epochs.get_data()[0, 0, np.where(epochs.times == t)[0][0]]
crop_method = epochs.copy().crop(tmin=t).get_data()[0, 0, 0]
get_data_method = epochs.get_data(tmin=t)[0, 0, 0]
print(f"Time matches sensor value -> we should see this value: {t}")
print(f"Direct indexing: {idx_method}")
print(f"crop(tmin/tmax) then get first: {crop_method}")
print(f"get_data(tmin/tmax): {get_data_method}")  # This one is different!!

Prints:

Time matches sensor value -> we should see this value: 0.77
Direct indexing:                0.7700000000000005
crop(tmin/tmax) then get first: 0.7700000000000005
get_data(tmin/tmax):            0.7599999999999998
Link to data

N/A

Expected results

I expect that epoch.get_data(tmin=t)[:,:,0] and epoch.crop(tmin=t).get_data()[:,:,0] are identical. Worse than this: if my dataset does have data at time t, I expect epoch.get_data(tmin=t)[:,:,0] to be the canonical way to get it, but it sometimes incorrectly gives me t-1 instead.

Actual results

See above.

Additional information

I first experienced this with mne version 1.7.0 but I tried my exemples with 1.12.0.dev66+g1138f8b25 and got the same results.

See also a more complete example below; in the real dataset I was working with the "off-by-one" corresponded to more than two thirds of the data–I believe it might have to do with cropping the data.

import mne
import numpy as np
import math
import pandas as pd

# Create a dataste, single sensor, 100Hzz, the sensor's value is time for
# easier debugging
data = np.linspace(-3.5, 1, 451).reshape((1,1,451))
epochs = mne.EpochsArray(data, mne.create_info(['test'], sfreq=100), tmin=-3.5, verbose=False)
assert np.allclose(epochs.times, epochs.get_data()[0,0]) # True by construction
# epochs.crop(tmin=-3.01, tmax=.99)
assert np.allclose(epochs.times, epochs.get_data()[0,0]) # True, crops preserves this OK

# simplest visible error:
t = 0.01
window = 0.011
idx_method = epochs.get_data()[0, 0, np.where(epochs.times == t)[0][0]]
crop_method = epochs.copy().crop(tmin=t, tmax=t+window).get_data()[0, 0, 0]
get_data_method = epochs.get_data(tmin=t, tmax=t+window)[0, 0, 0]

print(f"Direct indexing:     {idx_method}")
print(f"crop() then get:     {crop_method}")
print(f"get_data(tmin/tmax): {get_data_method}")
print(f"crop matches index?   {idx_method == crop_method}")
print(f"get_data matches index? {idx_method == get_data_method}")

print("\n" + ("="*55) + "\n")

# Let's collect some times/index/values lists
times, idxs, idvals, cvals, gvals = [], [], [], [], []
for relevant_time in epochs.times[2:-2]:
    times.append(relevant_time)
    idxs.append(np.where(epochs.times == relevant_time)[0][0])
    relevant_idx = idxs[-1]
    idvals.append(epochs.get_data()[0,0,relevant_idx])
    cvals.append(  epochs.copy().crop(tmin=relevant_time, tmax=relevant_time+.011).get_data()[0,0,0])
    gvals.append( epochs.get_data(    tmin=relevant_time, tmax=relevant_time+.011)[0,0,0])

# Just put this in a df for visualisation?
df = pd.DataFrame({"times": times,
     "idxs": idxs,
     "idvals": idvals,
     "cvals": cvals,
     "gvals": gvals,
     "match_c": [math.isclose(x, y) for  x,y in zip(idvals, cvals)],
     "match_g": [math.isclose(x, y) for  x,y in zip(idvals, gvals)],
     })

print(df)
assert np.mean(df.match_c) == 1
# assert np.mean(df.match_g) == 1  # Does not assert: mean is .676767 or 67/99
print(f"The two methods agree this fraction of the time: {np.mean(np.mean(df.match_g))}")

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the minimal example in the issue and inspect the EpochsArray get_data(tmin, tmax) and crop(tmin, tmax) entry points. Compare their handling of sample times and indices, then add coverage for the reported example so both methods return the same first sample when the requested time exists.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.