mne-tools / mne-tools/mne-python

RawArray._init_kwargs blocks garbage collection by retaining the input NumPy ndarray

Open
#14,074 7 comments 1 reaction 1 assignee View on GitHub

@orikanner is already working on this.

Since Jul 20, 2026.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

BaseRaw.__init__ stores constructor arguments in self._init_kwargs using _get_argvalues().

For RawArray, this includes the complete signal array:

raw._init_kwargs["data"] is raw._data  # True

After operations such as resample() replace raw._data, _init_kwargs["data"] continues retaining the original full-resolution array. Because raw.copy() uses a deep copy, the retained array is copied as well.

For large RawArray objects, this can keep several gigabytes of unused data alive. The cost can multiply when objects are copied or passed to worker processes, potentially causing severe memory pressure, memory failures, and poor parallel scaling.

How I encountered this

I found this while memory profiling a multi stage pipeline for high-sampling-rate signals, where the initial RawArray data can be multiple gigabytes.

After downsampling from 10 kHz to ~200 Hz, the active data became much smaller, but memory profiling showed that the original full resolution array was still retained.

Proposed fix

One possible narrow fix would be to remove the stored data argument locally in RawArray.__init__:

super().__init__(
    info,
    data,
    first_samps=(int(first_samp),),
    dtype=dtype,
    verbose=verbose,
)
self._init_kwargs.pop("data", None)

This leaves the active signal in self._data while avoiding changes to _get_argvalues() or other readers.

I would be glad to submit a PR with a regression test if this approach looks reasonable.

Steps to reproduce
import gc
import weakref

import mne
import numpy as np

mne.set_log_level("ERROR")

data = np.zeros((64, 1_000_000), dtype=np.float64)  # ~512 MB
data_ref = weakref.ref(data)

raw = mne.io.RawArray(
    data,
    mne.create_info(64, 10_000.0, "eeg"),
)

print("aliased:", raw._init_kwargs["data"] is raw._data)

del data
raw.resample(200)
gc.collect()

print("current _data:", round(raw._data.nbytes / 1e6), "MB")
print("retained data:", round(raw._init_kwargs["data"].nbytes / 1e6), "MB")
print("original collected:", data_ref() is None)

raw_copy = raw.copy()

print(
    "copied retained data:",
    round(raw_copy._init_kwargs["data"].nbytes / 1e6),
    "MB",
)
print(
    "same retained array:",
    raw_copy._init_kwargs["data"] is raw._init_kwargs["data"],
)
Link to data

No response

Expected results

After deleting the original data variable and replacing raw._data during resampling, the original full-resolution array should be released from memory.

In this example, RawArray should keep only the current data of about 10 MB, instead of also keeping the unused original array of about 512 MB.

Actual results
aliased: True

# Active downsampled data:
current _data: 10 MB

# Unused original array is still retained:
retained data: 512 MB
original collected: False

# copy() creates another copy of the retained array:
copied retained data: 512 MB
same retained array: False
Additional information

Python 3.13.8 (tags/v3.13.8:a15ae61, Oct 7 2025, 12:34:25) [MSC v.1944 64 bit (AMD64)]
CPU Intel(R) Core(TM) Ultra 7 258V (8 cores)
Memory 31.5 GiB

Core

  • mne 1.12.1 (latest release)
  • numpy 2.5.1 (unknown linalg bindings (threadpoolctl module not found: No module named 'threadpoolctl'))
  • scipy 1.18.0
  • matplotlib 3.11.1 (backend=tkagg)

Numerical (optional)

  • unavailable sklearn, numba, nibabel, nilearn, dipy, openmeeg, cupy, pandas, h5io, h5py

Visualization (optional)

  • unavailable pyvista, pyvistaqt, vtk, qtpy, ipympl, pyqtgraph, mne-qt-browser, ipywidgets, trame_client, trame_server, trame_vtk, trame_vuetify

Ecosystem (optional)

  • unavailable mne-bids, mne-nirs, mne-features, mne-connectivity, mne-icalabel, mne-bids-pipeline, neo, eeglabio, edfio, curryreader, mffpy, pybv, pymef, antio, defusedxml

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.