mne-tools / mne-tools/mne-python

Permutation test number of tails

Open
#11,511 25 comments 0 reactions 1 assignee View on GitHub

@kimcoco is already working on this.

Since Nov 13, 2023.

BUG sprint-2023
Dominant language
Python
Stars
3.5k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
100

Description

Description of the problem

The p-value produced by the function permutation_t_test does not depend on whether a one- or two-tailed test is requested. This is not the expected behaviour.

Steps to reproduce
import numpy as np
import mne.stats as stats

testData = np.atleast_2d(np.arange(5)).T

for thisTail in [0, 1]:
    _, p_values, _ = stats.permutation_t_test(testData, tail=thisTail)
    print('p-value: {}'.format(p_values[0]))
Link to data

No response

Expected results

The p-value for a one-tailed test should be about half the size of the p-value for a two-tailed test.

Actual results

The p-value for a one-tailed test and two-tailed test are the same.

Permuting 255 times (exact test)...
p-value: 0.0078125
Permuting 511 times (exact test)...
p-value: 0.0078125
Additional information

MNE version: 0.22.0
Platform: Windows-10
Python: 3.8.6

numpy: 1.19.5 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy: 1.6.0
matplotlib: 3.3.3 {backend=Qt5Agg}

sklearn: 0.24.0
numba: 0.52.0
nibabel: 3.2.1
nilearn: 0.7.0
dipy: 1.3.0
cupy: Not found
pandas: 1.2.0
mayavi: 4.7.2
pyvista: 0.27.4 {pyvistaqt=0.2.0, OpenGL 4.5.0 - Build 30.0.101.1692 via Intel(R) UHD Graphics 620}
vtk: 9.0.1
PyQt5: 5.12.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.