mne-tools / mne-tools/mne-python

update rank estimation in compute_source_psd()

Open
#11,901 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BUG
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

mne.minimum_norm.compute_source_psd() still uses "old-fashined" rank estimation and may have problems with maxfiltered data.
For example, the part

if pca:
        U, s, Vh = linalg.svd(K, full_matrices=False)
        rank = np.sum(s > 1e-8 * s[0])
        K = s[:rank] * U[:, :rank]
        Vh = Vh[:rank]
        logger.info("Reducing data rank %d -> %d" % (len(s), rank))
    else:
        Vh = None
    is_free_ori = inverse_operator["source_ori"] == FIFF.FIFFV_MNE_FREE_ORI

should use the estimated rank of the data.

Steps to reproduce
I used it like


stc_psd, evo_psd = mne.minimum_norm.compute_source_psd(
raw=raw, inverse_operator=invop, lambda2=1 / 9., method='MNE',
fmin=0., fmax=140., n_fft=60000, overlap=0.5, pca=True,
nave=1, bandwidth='hann', low_bias=True, return_sensor=True)

and for 6 our of my 29 subjects it produced

Reducing data rank 370 → 1

and indeed in the output all times courses (source space and sensor space) were the same.



### Link to data

_No response_

### Expected results

It should have used the rank after maxfilter (for MEG). 

### Actual results

Output include (for 6 out of 29 subjects)

Reducing data rank 370 → 1

and all time courses (source and sensor space) indeed are the same.

### Additional information

Platform             Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.17
Python               3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0]
Executable           /imaging/local/software/mne_python/mne1.4.0_1/bin/python3.10
CPU                  x86_64 (16 cores)
Memory               188.7 GB

Core
?? mne               1.4.0
?? numpy             1.21.6 (MKL 2022.2-Product with 16 threads)
?? scipy             1.10.1
?? matplotlib        3.7.1 (backend=QtAgg)
?? pooch             1.7.0
?? jinja2            3.1.2

Numerical (optional)
?? sklearn           1.2.2
?? numba             0.56.4
?? nibabel           5.1.0
?? nilearn           0.10.1
?? dipy              1.7.0
?? openmeeg          2.5.6
?? pandas            2.0.1
?? unavailable       cupy

Visualization (optional)
?? pyvista           0.39.0 (OpenGL 3.3 (Core Profile) Mesa 18.3.6 via llvmpipe (LLVM 3.4, 256 bits))
?? pyvistaqt         0.0.0
?? ipyvtklink        0.2.2
?? vtk               9.2.6
?? qtpy              2.3.1 (PyQt5=5.15.6)
?? ipympl            0.9.3
?? pyqtgraph         0.13.3
?? mne-qt-browser    0.5.0

Ecosystem (optional)
?? mne-bids          0.12
?? mne-nirs          0.5.0
?? mne-features      0.2.1
?? mne-connectivity  0.5.0
?? mne-icalabel      0.4
?? mne-bids-pipeline 1.2.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at compute_source_psd() and inspect the PCA/SVD rank block and how the input data rank is obtained. Compare the current K-based estimate with the rank after maxfiltering; done means the function uses that data rank and no longer collapses valid outputs to rank 1. Validate with the reported maxfiltered scenario and existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.