mne-tools / mne-tools/mne-python
Permutation test number of tails
@kimcoco is already working on this.
Since Nov 13, 2023.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Description of the problem
The p-value produced by the function permutation_t_test does not depend on whether a one- or two-tailed test is requested. This is not the expected behaviour.
Steps to reproduce
import numpy as np
import mne.stats as stats
testData = np.atleast_2d(np.arange(5)).T
for thisTail in [0, 1]:
_, p_values, _ = stats.permutation_t_test(testData, tail=thisTail)
print('p-value: {}'.format(p_values[0]))
Link to data
No response
Expected results
The p-value for a one-tailed test should be about half the size of the p-value for a two-tailed test.
Actual results
The p-value for a one-tailed test and two-tailed test are the same.
Permuting 255 times (exact test)...
p-value: 0.0078125
Permuting 511 times (exact test)...
p-value: 0.0078125
Additional information
MNE version: 0.22.0
Platform: Windows-10
Python: 3.8.6
numpy: 1.19.5 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy: 1.6.0
matplotlib: 3.3.3 {backend=Qt5Agg}
sklearn: 0.24.0
numba: 0.52.0
nibabel: 3.2.1
nilearn: 0.7.0
dipy: 1.3.0
cupy: Not found
pandas: 1.2.0
mayavi: 4.7.2
pyvista: 0.27.4 {pyvistaqt=0.2.0, OpenGL 4.5.0 - Build 30.0.101.1692 via Intel(R) UHD Graphics 620}
vtk: 9.0.1
PyQt5: 5.12.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.