mne-tools / mne-tools/mne-python
How to handle datasets with invalid info[meas_id][secs]?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
I'm woking with the ds000246 OpenNeuro dataset:
$ aws s3 sync --no-sign-request s3://openneuro.org/ds000246 ds000246
$ cd ds000246/sub-emptyroom/meg
Reading the data works as expected:
import mne
raw = mne.io.read_raw_ctf('sub-emptyroom_task-noise_run-01_meg.ds')
Writing thows an exception:
raw.save('/tmp/foo.fif')
Traceback:
RuntimeError Traceback (most recent call last)
<ipython-input-4-eb369e79ee42> in <module>
----> 1 raw.save('/tmp/foo.fif')
<decorator-gen-155> in save(self, fname, picks, tmin, tmax, buffer_size_sec, drop_small_buffer, proj, fmt, overwrite, split_size, split_naming, verbose)
~/Development/mne-python/mne/io/base.py in save(self, fname, picks, tmin, tmax, buffer_size_sec, drop_small_buffer, proj, fmt, overwrite, split_size, split_naming, verbose)
1379 "split_naming must be either 'neuromag' or 'bids' instead "
1380 "of '{}'.".format(split_naming))
-> 1381 _write_raw(fname, self, info, picks, fmt, data_type, reset_range,
1382 start, stop, buffer_size, projector, drop_small_buffer,
1383 split_size, split_naming, part_idx, None, overwrite)
~/Development/mne-python/mne/io/base.py in _write_raw(fname, raw, info, picks, fmt, data_type, reset_range, start, stop, buffer_size, projector, drop_small_buffer, split_size, split_naming, part_idx, prev_fname, overwrite)
1844
1845 picks = _picks_to_idx(info, picks, 'all', ())
-> 1846 fid, cals = _start_writing_raw(use_fname, info, picks, data_type,
1847 reset_range, raw.annotations)
1848
~/Development/mne-python/mne/io/base.py in _start_writing_raw(name, info, sel, data_type, reset_range, annotations)
2018 cals.append(info['chs'][k]['cal'] * info['chs'][k]['range'])
2019
-> 2020 write_meas_info(fid, info, data_type=data_type, reset_range=reset_range)
2021
2022 #
~/Development/mne-python/mne/io/meas_info.py in write_meas_info(fid, info, data_type, reset_range)
1453 """
1454 info._check_consistency()
-> 1455 _check_dates(info)
1456
1457 # Measurement info
~/Development/mne-python/mne/io/meas_info.py in _check_dates(info, prepend_error)
1411 if (value[key_2] < np.iinfo('>i4').min or
1412 value[key_2] > np.iinfo('>i4').max):
-> 1413 raise RuntimeError('%sinfo[%s][%s] must be between '
1414 '"%r" and "%r", got "%r"'
1415 % (prepend_error, key, key_2,
RuntimeError: info[meas_id][secs] must be between "-2147483648" and "2147483647", got "-5364633480"
How to best deal with data like this? Can I simply set info[meas_id][secs] to an arbitrary (valid) value? Also it seems a little odd that I can create (and work with) some data by reading it, but then cannot write it back to disk…
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the ds000246 CTF recording using mne.io.read_raw_ctf and raw.save. Read mne/io/base.py around _start_writing_raw and mne/io/meas_info.py around write_meas_info and _check_dates; done means agreeing on the supported handling for invalid info[meas_id][secs] and covering the read/write behavior with a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100