6 Sample Size

A sample is a subset of a population selected to represent the entire population in a study. It is used because while the study is interested in learning more about the population, it may not always be feasible to study every member of that population. The reasons can be, for example, unfeasible incurred costs11 For example in the form of time and money. or, it may just be impossible due to geographical distribution or availability.

6.1 Factors Affecting the Size of the Sample

For any study, the sample size depends on a few elements:

• Level of significance (\alpha), the error rate the researcher is ready to accept, such as \alpha = 0.05. It is the probability of rejecting the null hypothesis when it is true (a Type I error). It is set before the study, and it is not the same as the p-value, which is computed from the data22 See Hypothesis Testing..

  2 See Hypothesis Testing.

• Power, discussed in the context of Type II error, the failure to detect a difference when one does exist, or the chance of false negatives. The power of the study increases with the decrease in the chance of committing a Type II error. Usually 80% is an acceptable level for the power of a study. It means that the researcher accepts that the study misses a real difference of the expected size one time in five. For more strict studies, power can be increased to 90% or more.
• Expected effect size, the size of the difference or relationship the study should be able to detect, such as the difference between the means of two groups in standard deviation units. The smaller the effect, the larger the sample needed to detect it. Most often it is determined based on prior studies reported in the literature, or as the smallest effect that would matter in practice33 See Effect Size and Power..

  3 See Effect Size and Power.

• For a study that estimates a proportion, the expected proportion in the population, estimated from previous studies.
• For a study that estimates a mean, the population standard deviation, a measure of variability.

When estimating sample size a researcher should consider other elements as well, such as administrative issues, costs, possible participant response rate, and so forth. Each study should be considered from all angles and all potential elements that could participate in determining the sample should be studied carefully.

6.2 Methods of Determining the Sample Size

A cursory review of the literature shows that sample size can be determined in many ways using formulas and/or tables and that there is no universal “formula” for sample size calculations. Each of the methods has a recommended use.

For example, an approach to make a rough determination of a sample size for an experimental design using effect size and power was discussed in the Controlling Power Through Sample Size section. You will also find many sample size calculators available online, many of them based on Cochran’s Sample Size Formula.

6.2.1 Cochran’s Sample Size Formula

Used to compute an ideal sample size for a desired level of precision, it is recommended to be used for studies with infinite populations (Cochran 1977). It applies to studies that estimate a proportion from a random sample, such as the percentage of students who do something. It is not a power calculation: it tells how precisely a proportion is estimated, not whether a test will detect an effect. For the tests in this book, the sample size comes from a power analysis44 See Controlling Power Through Sample Size..

n_{0}=\dfrac{z^{2}\cdot p\cdot(1-p)}{e^{2}}

e: desired level of precision, the margin of error

p: the fraction of the population (as a proportion, between 0 and 1) that displays the attribute

z: the z-value, extracted from a z-table55 The entry for z in a z-table represents the area under the normal distribution curve to the left of z (Figure 6.1)..

Let’s consider an example. Think of a study of students in a large university campus for which we don’t know the campus size66 For example a large campus may have 10 - 15 K students. We are interested in finding the percentage of students who eat lunch at the campus dinner halls but we do not have insider information. The question is how many students would we need to ask that question to be able to determine, with reasonable confidence, what percentage of students conform to the sought behavior. Given the lack of information we start by considering that 50% of the students eat lunch at the school dining halls, which provides the largest variability. Then we consider a 95% confidence level (leading to an \alpha=0.05) and a ±5% precision. From the z-tables, the value for z is 1.96. Therefore, the theoretical sample would be:

n_{0}=\dfrac{1.96^{2}\cdot0.5\cdot(1-0.5)}{0.05^{2}}=384.16\approx385

How to find the value of z from a z-table. The procedure is:

1. Convert the confidence level from percent form to decimal form as value between 0 and 1. (95% \rightarrow 0.95)
2. Subtract the value from 1 and divide by 2 to find out how much is half (1 - 0.95 = 0.05; 0.05/2 = 0.025)
3. Add the value from 2) to the value from 1) (0.95 + 0.025 = 0.975)
4. Look for the value obtained in step 3) in table values. In Table 6.1 the value sits at the intersection of row labeled 1.9 and column labeled 0.06.
5. Determine the value of z by adding the value for the column with the value for the row obtained in step 4 (1.9 + 0.06 = 1.96).

6.2.2 Cochran’s Modified Formula for Finite Populations

A slightly modified formula can be used if the size of the population is known.

n=\dfrac{n_{0}}{1+\dfrac{n_{0}-1}{N}}

n₀: Cochran’s sample size computed using the formula for ideal sample size;

N: the size of the population77 The sample size is dependent on the size of the population until the population reaches about 40-50 K, after which the increase is almost none. Therefore, if the estimated population is this large or larger, the theoretical sample size, as computed for an unknown population, is about equal to the one generated by the modified formula..

As an example, let’s look at the same problem as before but for a much smaller campus of N = 600 students. While we can still use the theoretical sample of 385 participants computed before, do we need to? The necessary sample size may be smaller.

n=\dfrac{385}{1+\dfrac{385-1}{600}}=234.76\approx235

The result of this computation indicates that for smaller populations the number of subjects (sample size) can be smaller (235 vs. 385) for the researchers to be reasonably confident of the findings.

6.2.3 Yamane’s Simplified Formula for Sample Size

To make it simpler to compute the sample size without over estimating it when the population is known Yamane (1967) proposed the following formula, a shortcut of the same kind that assumes a proportion of .5 and a confidence level of about 95%:

n=\dfrac{N}{1+N\cdot e^{2}}

N - population size

e - level of precision

Using the same example as before, Yamane’s formula would suggest a sample size of 240 subjects for a student population of 600.

n=\dfrac{600}{1+600\cdot0.05^{2}}=240