PyPermut is a Python package implementing permutation tests, for statistics and machine learning.
PyPermut is distributed under the open source 3-clause BSD license.
In a nutshell, this package extends some parametric and non-parametric statistical tests into permutation tests for multivariate samples, allowing the computation of exact p-values and the correction for multiple tests.
Futhermore, this package offers several functions for machine learning, like a generic permutation test for any metric based on the predictions of a model (for eg, AUROC, AUPR, negative log-likelihood, etc).
This package implements several permutation tests (also called randomization tests).
Correlation tests:
Ordinal and numerical location tests:
In permutation tests, the null distribution is sampled by random permutations of measurements. Making no hypothesis on the statistic distribution, the p-value is defined as the proportion of the null distribution with test statistic greater (or, for certain test statistic, lower) than or equal to the test statistic of the observed data. When all permutations are evaluated, these permutation tests give exact tests.
In case of several variables, they naturally perform a correction for multiple tests:
In the context of correlated variables, they efficiently correct the family-wise error rate, being an alternative to the well-known Bonferroni correction which would lead to an undesirable lack of statistical power. These multiple variables can be useful to compute pairwise post-hoc tests.
See examples
.
Westfall and Young, Resampling-based multiple testing: examples and methods for p-value adjustment, Wiley-Blackwell, 1993.
Pesarin, Multivariate Permutation Tests – With Applications in Biostatistics, Wiley-Blackwell, 2001.
Nichols and Holmes, "Nonparametric permutation tests for functional neuroimaging: A primer with examples", Human Brain Mapping, vol 15, pp 1-25, 2001.
Edgington and Onghena, Randomization Tests, Fourth Edition, Chapman and Hall/CRC, 2007.
TODO: put documentation on RTD.
To install PyPermut as a standard module:
pip install path/to/pypermut
To install PyPermut in editable / development mode, in the folder:
pip install poetry
poetry install
Use pytest
.
Quentin Barthélemy, an anonymous researcher and Louis Mayaud, under the wisdom of Marco Congedo.