mne-tools / mne-tools/mne-python
How to deal with anonymization if one of the dates slips out of range?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Working with the somato dataset, I ran into an issue related to the fact that the measurement date I'm intending to anonymize deviates from the date in info['file_id']['secs'] such that, when I reach the desired adjustment for the measurement date, info['file_id']['secs'] goes out-of-range (even though the measurement date would be within the valid range).
MWE:
import mne
import os.path as op
from datetime import datetime
data_path = mne.datasets.somato.data_path()
raw_fname = op.join(data_path, 'sub-01', 'meg', 'sub-01_task-somato_meg.fif')
raw = mne.io.read_raw_fif(raw_fname, verbose='error')
print(f"Date in info['meas_date']: {raw.info['meas_date']}")
print(f"Date in info['file_id']: {datetime.fromtimestamp(raw.info['file_id']['secs'])}")
# Works
print('\nAnonymizing with daysback -> 10 years')
daysback = 10 * 365
raw_anon = raw.copy().anonymize(daysback=daysback)
print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
print(f"Date in info['file_id']: {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")
# Does not work
print('\nAnonymizing with daysback -> 90 years')
daysback = 90 * 365
raw_anon = raw.copy().anonymize(daysback=daysback)
print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
print(f"Date in info['file_id']: {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")
Output:
Date in info['meas_date']: 2007-07-05 11:17:11.172243+00:00
Date in info['file_id']: 1970-01-01 01:00:00
Anonymizing with daysback -> 10 years
Date in info['meas_date']: 1997-07-07 11:17:11.172243+00:00
Date in info['file_id']: 1960-01-04 01:35:47
Anonymizing with daysback -> 90 years
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Untitled-2 in
21 print('\nAnonymizing with daysback -> 90 years')
22 daysback = 90 * 365
---> 23 raw_anon = raw.copy().anonymize(daysback=daysback)
24 print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
25 print(f"Date in info['file_id']: {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")
in anonymize(self, daysback, keep_his, verbose)
~/Development/mne-python/mne/channels/channels.py in anonymize(self, daysback, keep_his, verbose)
593 .. versionadded:: 0.13.0
594 """
--> 595 anonymize_info(self.info, daysback=daysback, keep_his=keep_his,
596 verbose=verbose)
597 self.set_meas_date(self.info['meas_date']) # unify annot update
in anonymize_info(info, daysback, keep_his, verbose)
~/Development/mne-python/mne/io/meas_info.py in anonymize_info(info, daysback, keep_his, verbose)
2245 'daysback parameter was too large.'
2246 'Underlying Error:\n')
-> 2247 _check_dates(info, prepend_error=err_mesg)
2248
2249 return info
~/Development/mne-python/mne/io/meas_info.py in _check_dates(info, prepend_error)
1454 if (value[key_2] < np.iinfo('>i4').min or
1455 value[key_2] > np.iinfo('>i4').max):
-> 1456 raise RuntimeError('%sinfo[%s][%s] must be between '
1457 '"%r" and "%r", got "%r"'
1458 % (prepend_error, key, key_2,
RuntimeError: anonymize_info generated an inconsistent info object. daysback parameter was too large.Underlying Error:
info[file_id][secs] must be between "-2147483648" and "2147483647", got "-2838237853"
How shall I go about anonymizing this dataset? I want to use daysback large enough to move the measurement date before 1925, as is required by BIDS.
cc @agramfort
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with mne/io/meas_info.py, especially anonymize_info and _check_dates, then inspect the call from mne/channels/channels.py. Reproduce the failure with the somato dataset and the 90-year daysback example. Done means anonymization handles the differing meas_date and file_id dates without producing an invalid info object, while preserving the requested measurement-date adjustment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100