mne-tools / mne-tools/mne-python
In `crop(tmin)` versus `get_data(tmin)`, `tmin` has a different meaning
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Description of the problem
Hi,
Thanks again for the work MNE, I use it and recommend it all the time.
I was bitten recently by an off-by-one error that I ended up tracing back to the fact that epoch.crop(tmin) and epoch.get_data(tmin) interpret tmin differently. The easiest is probably to show a minimal working example below.
Thanks,
Steps to reproduce
Running the following code:
import mne
import numpy as np
# Create a datastet w/ single sensor, 100Hz, ramping value matching time
data = np.linspace(-3.5, 1, 451).reshape((1,1,451))
epochs = mne.EpochsArray(data, mne.create_info(['test'], 100), tmin=-3.5, verbose=False)
t = 0.77
idx_method = epochs.get_data()[0, 0, np.where(epochs.times == t)[0][0]]
crop_method = epochs.copy().crop(tmin=t).get_data()[0, 0, 0]
get_data_method = epochs.get_data(tmin=t)[0, 0, 0]
print(f"Time matches sensor value -> we should see this value: {t}")
print(f"Direct indexing: {idx_method}")
print(f"crop(tmin/tmax) then get first: {crop_method}")
print(f"get_data(tmin/tmax): {get_data_method}") # This one is different!!
Prints:
Time matches sensor value -> we should see this value: 0.77
Direct indexing: 0.7700000000000005
crop(tmin/tmax) then get first: 0.7700000000000005
get_data(tmin/tmax): 0.7599999999999998
Link to data
N/A
Expected results
I expect that epoch.get_data(tmin=t)[:,:,0] and epoch.crop(tmin=t).get_data()[:,:,0] are identical. Worse than this: if my dataset does have data at time t, I expect epoch.get_data(tmin=t)[:,:,0] to be the canonical way to get it, but it sometimes incorrectly gives me t-1 instead.
Actual results
See above.
Additional information
I first experienced this with mne version 1.7.0 but I tried my exemples with 1.12.0.dev66+g1138f8b25 and got the same results.
See also a more complete example below; in the real dataset I was working with the "off-by-one" corresponded to more than two thirds of the data–I believe it might have to do with cropping the data.
import mne
import numpy as np
import math
import pandas as pd
# Create a dataste, single sensor, 100Hzz, the sensor's value is time for
# easier debugging
data = np.linspace(-3.5, 1, 451).reshape((1,1,451))
epochs = mne.EpochsArray(data, mne.create_info(['test'], sfreq=100), tmin=-3.5, verbose=False)
assert np.allclose(epochs.times, epochs.get_data()[0,0]) # True by construction
# epochs.crop(tmin=-3.01, tmax=.99)
assert np.allclose(epochs.times, epochs.get_data()[0,0]) # True, crops preserves this OK
# simplest visible error:
t = 0.01
window = 0.011
idx_method = epochs.get_data()[0, 0, np.where(epochs.times == t)[0][0]]
crop_method = epochs.copy().crop(tmin=t, tmax=t+window).get_data()[0, 0, 0]
get_data_method = epochs.get_data(tmin=t, tmax=t+window)[0, 0, 0]
print(f"Direct indexing: {idx_method}")
print(f"crop() then get: {crop_method}")
print(f"get_data(tmin/tmax): {get_data_method}")
print(f"crop matches index? {idx_method == crop_method}")
print(f"get_data matches index? {idx_method == get_data_method}")
print("\n" + ("="*55) + "\n")
# Let's collect some times/index/values lists
times, idxs, idvals, cvals, gvals = [], [], [], [], []
for relevant_time in epochs.times[2:-2]:
times.append(relevant_time)
idxs.append(np.where(epochs.times == relevant_time)[0][0])
relevant_idx = idxs[-1]
idvals.append(epochs.get_data()[0,0,relevant_idx])
cvals.append( epochs.copy().crop(tmin=relevant_time, tmax=relevant_time+.011).get_data()[0,0,0])
gvals.append( epochs.get_data( tmin=relevant_time, tmax=relevant_time+.011)[0,0,0])
# Just put this in a df for visualisation?
df = pd.DataFrame({"times": times,
"idxs": idxs,
"idvals": idvals,
"cvals": cvals,
"gvals": gvals,
"match_c": [math.isclose(x, y) for x,y in zip(idvals, cvals)],
"match_g": [math.isclose(x, y) for x,y in zip(idvals, gvals)],
})
print(df)
assert np.mean(df.match_c) == 1
# assert np.mean(df.match_g) == 1 # Does not assert: mean is .676767 or 67/99
print(f"The two methods agree this fraction of the time: {np.mean(np.mean(df.match_g))}")
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimal example in the issue and inspect the EpochsArray get_data(tmin, tmax) and crop(tmin, tmax) entry points. Compare their handling of sample times and indices, then add coverage for the reported example so both methods return the same first sample when the requested time exists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 62/100