mne-tools / mne-tools/mne-python
raw.crop failure with multi-file CTF
@massich is already working on this.
Since Jul 22, 2019.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
I'm unsure about the exact meaning of raw.first_samp, but I think there might be an error when cropping multi-file CTF recordings.
I'm working with ~1h long CTF recordings that I crop into blocks to preprocess. Unfortunately the CTF recordings is big (>10GB), so hard to share. The following demonstrates the problem:
>>> import mne
>>> raw = mne.io.read_raw_ctf(filename)
ds directory : /home/gortega/megdata/S1-5_Attractor_20161022_01.ds
res4 data read.
hc data read.
Separate EEG position data file read.
Quaternion matching (desired vs. transformed):
-1.14 80.92 0.00 mm <-> -1.14 80.92 0.00 mm (orig : -53.00 56.58 -262.06 mm) diff = 0.000 mm
1.14 -80.92 0.00 mm <-> 1.14 -80.92 0.00 mm (orig : 48.86 -69.11 -267.17 mm) diff = 0.000 mm
100.30 0.00 0.00 mm <-> 100.30 0.00 0.00 mm (orig : 75.61 54.06 -244.96 mm) diff = 0.000 mm
Coordinate transformations established.
Polhemus data for 3 HPI coils added
Device coordinate locations for 3 HPI coils added
64 EEG electrode locations assigned to channel info.
64 EEG locations added to Polhemus data.
Measurement info composed.
Finding samples for /home/gortega/megdata/S1-5_Attractor_20161022_01.ds/S1-5_Attractor_20161022_01.meg4:
System clock channel is available, checking which samples are valid.
1104 x 1200 = 1324800 samples from 405 chs
Finding samples for /home/gortega/megdata/S1-5_Attractor_20161022_01.ds/S1-5_Attractor_20161022_01.1_meg4:
System clock channel is available, checking which samples are valid.
1104 x 1200 = 1324800 samples from 405 chs
Finding samples for /home/gortega/megdata/S1-5_Attractor_20161022_01.ds/S1-5_Attractor_20161022_01.2_meg4:
System clock channel is available, checking which samples are valid.
1104 x 1200 = 1324800 samples from 405 chs
Finding samples for /home/gortega/megdata/S1-5_Attractor_20161022_01.ds/S1-5_Attractor_20161022_01.3_meg4:
System clock channel is available, checking which samples are valid.
1104 x 1200 = 1324800 samples from 405 chs
Finding samples for /home/gortega/megdata/S1-5_Attractor_20161022_01.ds/S1-5_Attractor_20161022_01.4_meg4:
System clock channel is available, checking which samples are valid.
726 x 1200 = 871677 samples from 405 chs
723 samples omitted at the end
Current compensation grade : 0
There are 5 meg4 files for this dataset and croping beyond the first meg4 file doesn't work:
# Crop within first meg4 file:
raw.copy().crop((100+1)/1200, (100+10)/1200).first_samp
101 # This is OK!
# Crop in 2nd meg4 file:
>>> raw.copy().crop((1324800+1)/1200, (1324800+10)/1200).first_samp
1 # I believe this should be 1324800+1
I think the source of the problem is that raw._first_samps and raw._last_samps are not set correctly:
>>> raw._first_samps
array([0, 0, 0, 0, 0])
>>> raw._last_samps
array([1324799, 1324799, 1324799, 1324799, 871676])
Fixing them like so solves the problem:
>>> raw._first_samps = np.cumsum(raw._raw_lengths) - raw._raw_lengths[0]
>>> raw._last_samps = np.cumsum(raw._last_samps)
>>> raw.copy().crop((1324800+1)/1200, (1324800+10)/1200).first_samp
1324801 # OK!
But this leads to problems for me down the line.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.