mne-tools / mne-tools/mne-python
Curious behavior of FIFF anonymize and show_fiff
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Description of the problem
When anonymizing a file raw.anonymize() I've noted the following.
This is not important, the relevant data is anonymized, so I guess it is curious (?) to discuss.
Perhaps I'm doing something wrong? Maybe someone can replicate.
If is should be corrected I can fix it.
The data in the file_id tags are changed as intended (to 946684800=January 1, 2000 12:00:00 AM GMT)
The measurement date in the Measurement date tag (meas_date:204) is also changed correctly.
However, the dates/times in other tags (file_id, block_id, block_start, block_end) are off (by one second!) (ha!) And in case of block_start it goes to the epoch zero.
Related to the block_start tag too, there seems to be another issue when exploring the contents of a fiff file with mne.io.show_fiff.
Thanks.
Steps to reproduce
data_folder = mne.datasets.sample.data_path()
sample_raw_file = (
data_folder / "MEG" / "sample" / "sample_audvis_raw.fif"
)
raw = mne.io.read_raw_fif(sample_data_raw_file)
raw.anonymize()
#And the (maybe) related other issue:
print(mne.io.show_fiff(sample_data_raw_file))
Link to data
No response
Expected results
Not sure if a change is needed.
See Actual results for clarity.
Actual results
I think the date anonymizing algorithm is not taking into account that the date/time in each tag is the time of creation of the id tag, which can be a some seconds later, depending on each case. Thus, when subtracting a fixed value delta_t that offset gets translated to the final "anonymized" date. See https://github.com/mne-tools/mne-python/blob/a3e98cea745b75665b7692161629937f6dc65921/mne/io/meas_info.py#L2964
Apart from that, there seems to be a different problem when reading the date/time stamp on the block_start tag. Both the machine_id (which is usually the mac address of one network card) and the "anonymized" time stamp seem off. I can't find a reason for this behavior.
Overall, the result is the following. If sample_auidvis_raw.fif is anonymized, and you print the contents of tags containing time_stamps:
file_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID tag: 01.01.2000 12:00:00
block_id:
MAC address in ID tag changed: 30:33:30:33:30:33:30:33 <---- ?
Measurement date in ID tag changed: 01.01.1970 12:00:00 <---- it goes to zero!
block_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID: 31.12.1999 11:59:59 <------
block_id:
MAC address in ID tag: 00:00:00:00:00:00:00:00
Measurement date in ID: 31.12.1999 11:59:59 <------
meas_date:
Measurement date changed: 01.01.2000 12:00:00
Related to the other issue with block_start tag. When running print(mne.io.show_fiff(fiff_file)) I show:
999 = FIFFB_ROOT
100 = FIFF_FILE_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
101 = FIFF_DIR_POINTER (4b >i4) = [-1]
106 = FIFF_FREE_LIST (4b >i4) = [-1]
108 = FIFF_NOP (0b nul)
100 = FIFFB_MEAS <------ should be 104 = FIFF_BLOCK_START (4b ids) = {... with data = 100 (FIFFB_MEAS)
103 = FIFF_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
110 = FIFF_PARENT_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
101 = FIFFB_MEAS_INFO
110 = FIFF_PARENT_BLOCK_ID (20b ids) = {'version': 65540, 'machid': a ... dict len=4
212 = FIFF_EXPERIMENTER (13b str) = mne_anonymize ... str len=13
206 = FIFF_COMMENT/FIFF_DESCRIPTION (60b str) = Anonymized using a time shift ... str len=60
500 = FIFF_PROJ_ID (4b >i4) = [0]
501 = FIFF_PROJ_NAME (13b str) = mne_anonymize ... str len=13
204 = FIFF_MEAS_DATE (8b >i4) = [946684800 0] ... array size=2
Additional information
Not sure if these two to behaviors are related.
I'm running up to date from master branch (a3e98cea745b75665b7692161629937f6dc65921) on Python3.9 for MacOs (intel).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the anonymization logic in mne/io/meas_info.py near the referenced line and reproduce the timestamps using the sample FIFF file and raw.anonymize(). Then inspect mne.io.show_fiff on the same file to separate the timestamp-shift behavior from the block_start display issue. Done means the expected handling of file_id, block_id, block_start, block_end, and show_fiff output is established and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100