mne-tools / mne-tools/mne-python

Curious behavior of FIFF anonymize and show_fiff

Open
#11,795 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

When anonymizing a file raw.anonymize() I've noted the following.
This is not important, the relevant data is anonymized, so I guess it is curious (?) to discuss.
Perhaps I'm doing something wrong? Maybe someone can replicate.
If is should be corrected I can fix it.

The data in the file_id tags are changed as intended (to 946684800=January 1, 2000 12:00:00 AM GMT)
The measurement date in the Measurement date tag (meas_date:204) is also changed correctly.
However, the dates/times in other tags (file_id, block_id, block_start, block_end) are off (by one second!) (ha!) And in case of block_start it goes to the epoch zero.

Related to the block_start tag too, there seems to be another issue when exploring the contents of a fiff file with mne.io.show_fiff.

Thanks.

Steps to reproduce
data_folder = mne.datasets.sample.data_path()
sample_raw_file = (
    data_folder / "MEG" / "sample" / "sample_audvis_raw.fif"
)
raw = mne.io.read_raw_fif(sample_data_raw_file)
raw.anonymize()

#And the (maybe) related other issue:
print(mne.io.show_fiff(sample_data_raw_file))
Link to data

No response

Expected results

Not sure if a change is needed.
See Actual results for clarity.

Actual results

I think the date anonymizing algorithm is not taking into account that the date/time in each tag is the time of creation of the id tag, which can be a some seconds later, depending on each case. Thus, when subtracting a fixed value delta_t that offset gets translated to the final "anonymized" date. See https://github.com/mne-tools/mne-python/blob/a3e98cea745b75665b7692161629937f6dc65921/mne/io/meas_info.py#L2964

Apart from that, there seems to be a different problem when reading the date/time stamp on the block_start tag. Both the machine_id (which is usually the mac address of one network card) and the "anonymized" time stamp seem off. I can't find a reason for this behavior.
Overall, the result is the following. If sample_auidvis_raw.fif is anonymized, and you print the contents of tags containing time_stamps:

file_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID tag: 01.01.2000 12:00:00

block_id:
MAC address in ID tag changed: 30:33:30:33:30:33:30:33 <---- ?
Measurement date in ID tag changed: 01.01.1970 12:00:00 <---- it goes to zero!

block_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID: 31.12.1999 11:59:59 <------

block_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID: 31.12.1999 11:59:59 <------

meas_date:
Measurement date changed: 01.01.2000 12:00:00

Related to the other issue with block_start tag. When running print(mne.io.show_fiff(fiff_file)) I show:

999 = FIFFB_ROOT
    100 = FIFF_FILE_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
    101 = FIFF_DIR_POINTER (4b >i4) = [-1]
    106 = FIFF_FREE_LIST (4b >i4) = [-1]
    108 = FIFF_NOP (0b nul)
    100 = FIFFB_MEAS            <------ should be 104 = FIFF_BLOCK_START (4b ids) = {... with data = 100 (FIFFB_MEAS)
        103 = FIFF_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
        110 = FIFF_PARENT_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
        101 = FIFFB_MEAS_INFO
            110 = FIFF_PARENT_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
            212 = FIFF_EXPERIMENTER (13b str) = mne_anonymize ... str len=13
            206 = FIFF_COMMENT/FIFF_DESCRIPTION (60b str) = Anonymized using a time shift  ... str len=60
            500 = FIFF_PROJ_ID (4b >i4) = [0]
            501 = FIFF_PROJ_NAME (13b str) = mne_anonymize ... str len=13
            204 = FIFF_MEAS_DATE (8b >i4) = [946684800         0] ... array size=2
Additional information

Not sure if these two to behaviors are related.
I'm running up to date from master branch (a3e98cea745b75665b7692161629937f6dc65921) on Python3.9 for MacOs (intel).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the anonymization logic in mne/io/meas_info.py near the referenced line and reproduce the timestamps using the sample FIFF file and raw.anonymize(). Then inspect mne.io.show_fiff on the same file to separate the timestamp-shift behavior from the block_start display issue. Done means the expected handling of file_id, block_id, block_start, block_end, and show_fiff output is established and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.