mne-tools / mne-tools/mne-python

MNE appends empty data at the end of Raw object after CNT importation

Open
#11,802 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

Hello! My lab has some old neuroscan cnt files that we are processing and I discovered a bug where MNE appends empty data to the end of the file after importation using mne.io.read_raw_cnt()

When I read the file in neuroscan the file is normal so I know it is not a problem with the file.

Steps to reproduce
import mne
mne.io.read_raw_cnt('empty_end.cnt', data_format = 'int32').plot()
Link to data

I cannot shorten the file because it removes the problem.
Here is a link to the file:
https://wustl.box.com/s/nfcbstcegtbh954ixh8orzyteiz0wxk2

Expected results

The expected results should be the data imports as normal and looks like this at the end:
image

Actual results

Instead there is empty data at the end and when I run:
mne.io.read_raw_cnt('/home/woess/current_assignment/empty_end.cnt', data_format = 'int32').to_data_frame().tail(100)
The data looks like this:
image
The plot looks like this:
image

Additional information

Platform Linux-5.15.90.2-microsoft-standard-WSL2-x86_64-with-glibc2.31
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0]
Executable /home/woess/mambaforge/envs/mne/bin/python
CPU x86_64 (12 cores)
Memory 11.3 GB

Core
├☑ mne 1.4.0
├☑ numpy 1.23.5 (OpenBLAS 0.3.21 with 12 threads)
├☑ scipy 1.10.1
├☑ matplotlib 3.7.1 (backend=module://matplotlib_inline.backend_inline)
├☑ pooch 1.7.0
└☑ jinja2 3.1.2

Numerical (optional)
├☑ sklearn 1.2.2
├☑ numba 0.56.4
├☑ nibabel 5.0.1
├☑ nilearn 0.10.0
├☑ dipy 1.6.0
├☑ openmeeg 2.5.6
├☑ pandas 2.0.3
└☐ unavailable cupy

Visualization (optional)
├☑ pyvista 0.38.5 (OpenGL 3.3 (Core Profile) Mesa 21.2.6 via D3D12 (AMD Radeon(TM) Graphics))
├☑ pyvistaqt 0.9.1
├☑ ipyvtklink 0.2.2
├☑ vtk 9.2.6
├☑ qtpy 2.3.0 (PyQt5=5.15.6)
├☑ ipympl 0.9.3
├☑ pyqtgraph 0.13.2
└☑ mne-qt-browser 0.4.0

Ecosystem (optional)
└☐ unavailable mne-bids, mne-nirs, mne-features, mne-connectivity, mne-icalabel, mne-bids-pipeline

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the mne.io.read_raw_cnt() entry point and reproduce the issue with the linked empty_end.cnt file using data_format='int32'. Inspect how CNT sample counts and trailing records are read; done means the imported Raw object and to_data_frame().tail(100) contain no appended empty data and the end matches the expected plot.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.