mne-tools / mne-tools/mne-python

How to deal with anonymization if one of the dates slips out of range?

Open
#8,128 32 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Working with the somato dataset, I ran into an issue related to the fact that the measurement date I'm intending to anonymize deviates from the date in info['file_id']['secs'] such that, when I reach the desired adjustment for the measurement date, info['file_id']['secs'] goes out-of-range (even though the measurement date would be within the valid range).

MWE:

import mne
import os.path as op
from datetime import datetime

data_path = mne.datasets.somato.data_path()
raw_fname = op.join(data_path, 'sub-01', 'meg', 'sub-01_task-somato_meg.fif')

raw = mne.io.read_raw_fif(raw_fname, verbose='error')
print(f"Date in info['meas_date']: {raw.info['meas_date']}")
print(f"Date in info['file_id']:   {datetime.fromtimestamp(raw.info['file_id']['secs'])}")

# Works
print('\nAnonymizing with daysback -> 10 years')
daysback = 10 * 365
raw_anon = raw.copy().anonymize(daysback=daysback)
print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
print(f"Date in info['file_id']:   {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")

# Does not work
print('\nAnonymizing with daysback -> 90 years')
daysback = 90 * 365
raw_anon = raw.copy().anonymize(daysback=daysback)
print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
print(f"Date in info['file_id']:   {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")

Output:

Date in info['meas_date']: 2007-07-05 11:17:11.172243+00:00
Date in info['file_id']:   1970-01-01 01:00:00

Anonymizing with daysback -> 10 years
Date in info['meas_date']: 1997-07-07 11:17:11.172243+00:00
Date in info['file_id']:   1960-01-04 01:35:47

Anonymizing with daysback -> 90 years
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Untitled-2 in 
     21 print('\nAnonymizing with daysback -> 90 years')
     22 daysback = 90 * 365
---> 23 raw_anon = raw.copy().anonymize(daysback=daysback)
     24 print(f"Date in info['meas_date']: {raw_anon.info['meas_date']}")
     25 print(f"Date in info['file_id']:   {datetime.fromtimestamp(raw_anon.info['file_id']['secs'])}")

 in anonymize(self, daysback, keep_his, verbose)

~/Development/mne-python/mne/channels/channels.py in anonymize(self, daysback, keep_his, verbose)
    593         .. versionadded:: 0.13.0
    594         """
--> 595         anonymize_info(self.info, daysback=daysback, keep_his=keep_his,
    596                        verbose=verbose)
    597         self.set_meas_date(self.info['meas_date'])  # unify annot update

 in anonymize_info(info, daysback, keep_his, verbose)

~/Development/mne-python/mne/io/meas_info.py in anonymize_info(info, daysback, keep_his, verbose)
   2245                 'daysback parameter was too large.'
   2246                 'Underlying Error:\n')
-> 2247     _check_dates(info, prepend_error=err_mesg)
   2248 
   2249     return info

~/Development/mne-python/mne/io/meas_info.py in _check_dates(info, prepend_error)
   1454                 if (value[key_2] < np.iinfo('>i4').min or
   1455                         value[key_2] > np.iinfo('>i4').max):
-> 1456                     raise RuntimeError('%sinfo[%s][%s] must be between '
   1457                                        '"%r" and "%r", got "%r"'
   1458                                        % (prepend_error, key, key_2,

RuntimeError: anonymize_info generated an inconsistent info object. daysback parameter was too large.Underlying Error:
info[file_id][secs] must be between "-2147483648" and "2147483647", got "-2838237853"

How shall I go about anonymizing this dataset? I want to use daysback large enough to move the measurement date before 1925, as is required by BIDS.

cc @agramfort

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with mne/io/meas_info.py, especially anonymize_info and _check_dates, then inspect the call from mne/channels/channels.py. Reproduce the failure with the somato dataset and the 90-year daysback example. Done means anonymization handles the differing meas_date and file_id dates without producing an invalid info object, while preserving the requested measurement-date adjustment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.