Power Analysis

The power analysis component of the statsmodels python library was developed to match part of the pwr reference library in the R statistical programming language. The computations are working with the following major distributions: normal, non-central t, non-central F, and non-central chi-square. This tool offers five computation models. Choose the one that matches your study.

The tool is not intended to be comprehensive, but as a starting point in your design. For more comprehensive options consider the R pwr package or the G*Power Software developed at the University of Dusseldorf.

The parameters required for the analysis may differ between the different types of analyses. Below is a quick overview of their meaning if the field names are insufficient.

IMPORTANT: For all analysis, one and only one field should be blank; the field that you want to calculate the value for. For example, if you have a limited pool of subjects, meaning that you already know the sample size, you can compute the Standardized effect size by entering a value in the Number of observations field, and leaving the effect size field blank.

Parameters

Standardized effect size: How large the effect is, in standard units. The t-tests and the z-test use Cohen's d, the difference between the two means divided by the standard deviation. For the paired samples t-test, d is the mean of the differences divided by the standard deviation of the differences, not of the scores; the two can give very different sample sizes. The ANOVA uses Cohen's f and the chi-square test uses Cohen's w. The value has to be positive. Can be greater than 1.

Number of observations (sample size): This is the size of your sample, for example the number of participants in your study. For a paired design it is the number of pairs. Repeated readings from the same participant are not separate observations. The value has to be at least 2.

Number of observations (sample size) for sample 1: In a two sample design, the size of the first sample. When it is computed, the size of the second sample is shown with it. The calculation for the t-test assumes equal variances in the two groups (Student's t-test). It is close for Welch's version, which the book uses, when the two groups have similar sizes and variances.

Ratio size of sample 2 / size of sample 1 (default=1): The ratio of the number of observations in sample 2 relative to sample 1. Has to be greater than 0.

Significance Level: The probability of a Type I error occurring. That is, the probability of wrong rejections if the Null Hypothesis is true. E.g. 0.05, the most widely used value. Has to be between 0 and 1.

Power of the test: Represents the probability that the test correctly rejects the Null Hypothesis if the Alternative Hypothesis is True. It is calculated as one minus the probability of Type II error. The value has to be between 0 and 1 (e.g., 0.8).

Alternative computations: Is an extra argument used to select if the power calculation is conducted for a two-sided (default) or a one-sided test.

Number of bins or cells in the distribution: Associated with the Chi-Square goodness of fit test, represents the number of groups (categories, bins) the data is divided into.

Number of groups: Associated with the one factor ANOVA, the number of groups being compared (at least 2). The sample size for the ANOVA is the total across all groups. The calculation assumes groups of equal size, so the computed total is rounded up to a number that splits evenly, and the size of each group is shown with it.