mne-tools / mne-tools/mne-python

2-Sample Permutation test

Open
#8,832 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ENH
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Describe the new feature or enhancement

Please provide a clear and concise description of what you want to add or
change.

Hi, I would like to add a 2 sample permutation T-test to MNE. I was in a rush a while back and was unable to use the permutation_cluster_test because I could not get an adjacency matrix with the data I had. Other permutation tests were all one sample. I tried using other packages, but realized all the statistical packages for python that had a permutation test ran into one or more problems: couldn't handle nan values well, weren't invariant to the number of dimensions, or had very strict array requirements. Here is the test I threw together at the time: https://github.com/Winter-Bot/2sample_Permutation_ttest . It takes into account nan values in the ttest, it is able to take data arrays from 2D to 4D, and it can take pure arrays or a list of arrays, so long as the first dimension is the observations/participants.

Describe your proposed implementation

Describe how you think the feature or improvement should be implemented (e.g.,
as a new method on an existing class? as new capability added to an existing
method?) If you're not sure, please delete this section and the next section.

I was wondering if you wanted me to fork it. Or you could take it and modify it, adding it to other existing code. That is okay too.
https://github.com/Winter-Bot/2sample_Permutation_ttest

Describe possible alternatives

If you've suggested an implementation above, list here any alternative
implementations you can think of, and brief comments explaining why the chosen
implementation is better.

forked to mne.stats. or included into one of the other existing permutation test.

Additional comments

Add any other context or screenshots about the feature request here.
I tested my code against these:
(1) nnstats.permtest_rel() (Network Neuroscience Lab at McGill University, 2018)
(2) mlxtend.evaluate.permutation_test() function from Raschka, Sebastian (2018)
(3) mne.stats.permutation_cluster_test() function (Gramfort et al., 2014)
(4) permutation_test() functions from Python Software Foundation (2017)
with HCP-MEG eravg data and tfavg data. I did a short write up if you want to see it let me know.
I have it give several types of outputs, each one being more or less applicable depending on the type of data being input. If you want to take it and completely reformulate it that is fine too. If you have questions for me or need me to change anything on it, you can reach me at m.a.winter87@gmail.com

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing permutation-test entry points in mne.stats, especially permutation_cluster_test, and compare them with the linked 2sample_Permutation_ttest implementation. Clarify the public API and expected behavior for two samples, NaN values, multidimensional arrays, and supported input forms; the work is done when the agreed capability is integrated with verification for those cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.