So far, we have used descriptive statistics to describe a sample of data by calculating sample statistics such as \(\bar{x}\) (the sample mean) and \(s\) (the sample standard deviation).
However, research is often conducted with the aim of using these sample statistics to estimate and compare true values of a population. These true values are called population parameters and are represented by Greek letters such as \(\mu\) (the population mean) and \(\sigma\) (the population standard deviation). Inferential statistics allow us to make statements about unknown population parameters using sample statistics from a random sample of the population. There are two main types of inferential statistics, estimation and hypothesis testing, and both are covered on this page.
In brief, it covers the following:
First, though, you may like to test your understanding of inferential statistics by choosing the best answer to the following question:
When sample statistics are used to estimate population parameters, this is known as estimation. Estimation can be done using either a single value, called a point estimate, or a range of values, called a confidence interval. These two types of estimation are explained in the following sections.
Suppose a researcher is interested in cholesterol levels in a population. If they select a random sample from that population, they can use statistics calculated from the sample to estimate the corresponding population parameters. In this case:
While point estimates are useful, it is often better to estimate a population parameter using a range of values. This helps account for the fact that sample statistics are unlikely to be exactly the same as the corresponding population parameters. This is where confidence intervals come in.
A confidence interval gives a range of values that can be used to estimate a population parameter. It also includes a confidence coefficient, which shows how certain we are that the interval contains the population parameter. This is usually set at \(95\%\). For example, a \(95\%\) confidence interval for the population mean blood glucose level of (\(4\) mmol/L, \(6\) mmol/L) indicates that we are \(95\%\) certain that the population mean blood glucose level lies between \(4\) mmol/L and \(6\) mmol/L.
For another way of interpreting confidence intervals, recall the concept of sampling distributions and imagine calculating a confidence interval for every possible sample of the same size. A \(95\%\) confidence interval means that you would expect 95 out of every 100 of these intervals to contain the population mean.
While confidence intervals can be calculated for many different statistics, a common example is a confidence interval for the population mean. This can be calculated based on the sampling distribution of the sample mean using the following formula:
\[\textrm{confidence interval for population mean} = \textrm{sample mean} \pm \textrm{a multiple of the standard error of the mean}\]There are two important things to note about this formula. First, the multiple depends on the confidence coefficient. For example, a \(95\%\) confidence interval requires a multiple of \(1.96\). This relates to the normal distribution, where approximately \(95\%\) of the area under the normal curve lies within 1.96 standard deviations of the mean. Second, the standard error of the mean for a sample of size \(n\) with standard deviation \(s\) is equal to \(\frac{s}{\sqrt{n}}\)
Therefore, the formula for a \(95\%\) confidence interval for a population mean, based on a sample of size \(n\) with mean \(\bar{x}\) and standard deviation \(s\), is:
\[95\% \textrm{ confidence interval for population mean} = \bar{x} \pm 1.96 \times \frac{s}{\sqrt{n}}\]Before calculating a confidence interval for a population mean, you should make sure the following assumptions are met:
Assumption 1: The sample is a random sample that is representative of the population.
Assumption 2: The observations are independent. This means that measurements for one participant do not affect the measurements of another participant.
Assumption 3: The variable is normally distributed, or the sample size is large enough that the sampling distribution of the mean approximates a normal distribution.
Some important points about confidence intervals are:
Finally, remember that a population parameter is fixed. It is the sample statistic and confidence interval that change from sample to sample. Once a confidence interval has been calculated, the population parameter is either inside or outside the interval. We can only state the level of certainty that we believe the interval contains the population parameter.
If you would like to practise calculating and interpreting confidence intervals, have a go at one or both of the following activities.
Hypothesis testing involves developing hypotheses about a population based on information observed in a sample. These hypotheses can then be tested to determine whether differences or relationships observed in the sample are statistically significant for the population, or whether they may have occurred due to random chance in the sample.
To do this, two complementary but contradictory hypotheses need to be developed. These are called the null hypothesis and the alternative hypothesis (or research hypothesis).
The hypotheses are formulated differently depending on whether you are conducting a two-tailed or one-tailed hypothesis test. A two-tailed test tests for an effect in either direction, while a one-tailed test only tests for an effect in one direction.
The choice between a one-tailed and two-tailed test will depend on your methodology. However, two-tailed tests are typically used unless there is a reason to use a one-tailed test, such as when the effect is only possible in one direction. For this reason, this module will focus on two-tailed hypothesis tests. These tests use the following hypotheses:
Null hypothesis (\(\textrm{H}_\textrm{0}\)): This hypothesis states that there is no difference or relationship between variables in a population. For example, no significant difference between two population means, no significant association between two categorical variables, no significant correlation between two continuous variables or no significant difference from the normal distribution (as in the Shapiro-Wilk test).
Alternative hypothesis (\(\textrm{H}_\textrm{A}\)): Also known as the research hypothesis, this hypothesis states the opposite of the null hypothesis. It states that there is a difference or relationship between variables in a population. For example, that there is a significant difference between two population means, a significant association between two categorical variables, a significant correlation between two continuous variables or a significant difference from the normal distribution (as in the Shapiro-Wilk test).
Both hypotheses can be written using words or symbols, and there are often several ways to write them. For example, suppose we want to test whether there is a significant change in the mean blood pressure of a population of patients after they have taken a new medication. We could write the null and alternative hypotheses in several different ways:
\(\textrm{H}_\textrm{0}\): there is no significant difference in mean blood pressure before and after the medication
\(\textrm{H}_\textrm{0}: \mu_{\textrm{bp before}} = \mu_{\textrm{bp after}}\)
\(\textrm{H}_\textrm{0}: \mu_{\textrm{bp before}} - \mu_{\textrm{bp after}} = 0\)
\(\textrm{H}_\textrm{A}\): there is a significant difference in mean blood pressure before and after the medication
\(\textrm{H}_\textrm{A}: \mu_{\textrm{bp before}} \neq \mu_{\textrm{bp after}}\)
\(\textrm{H}_\textrm{A}: \mu_{\textrm{bp before}} - \mu_{\textrm{bp after}} \neq 0\)
If you would like to practise writing hypotheses, have a go at formulating null and alternative hypotheses for the following activity.
\(\textrm{H}_\textrm{0}\): There is no significant difference in mean heart rate before and after the fun run (\(\mu_{\textrm{hr before}} = \mu_{\textrm{hr after}}\), or \(\mu_{\textrm{hr before}} - \mu_{\textrm{hr after}} = 0\))
\(\textrm{H}_\textrm{A}\): There is a significant difference in mean heart rate before and after the fun run (\(\mu_{\textrm{hr before}} \neq \mu_{\textrm{hr after}}\), or \(\mu_{\textrm{hr before}} - \mu_{\textrm{hr after}} \neq 0\))
\(\textrm{H}_\textrm{0}\): There is no significant difference in mean grades for male and female students (\(\mu_{\textrm{male}} = \mu_{\textrm{female}}\), or \(\mu_{\textrm{male}} - \mu_{\textrm{female}} = 0\))
\(\textrm{H}_\textrm{A}\): There is a significant difference in mean grades for male and female students (\(\mu_{\textrm{male}} \neq \mu_{\textrm{female}}\), or \(\mu_{\textrm{male}} - \mu_{\textrm{female}} \neq 0\))
\(\textrm{H}_\textrm{0}\): There is no significant correlation between hours of study and exam marks (\(r = 0\))
\(\textrm{H}_\textrm{A}\): There is a significant correlation between hours of study and exam marks (\(r \neq 0\))
Once the hypotheses have been formulated, they can be tested to evaluate statistical significance, as explained in the following section. It is also important to consider practical significance, which is explained in the section after that.
To evaluate statistical significance, you need to conduct an appropriate statistical test. Common examples include one sample, paired samples and independent samples \(t\) tests, one-way ANOVA, the chi-square test of independence and Pearson’s correlation. All of these tests are covered on later pages of this module. You will typically conduct these tests using statistical software or a programming language, such as SPSS, Stata, SAS, R or Python. However, they can also be calculated manually, although this is not covered here.
Whichever test you use, it will calculate a test statistic. This compares the value of the sample statistic (for example, the sample mean change in blood pressure in our blood pressure example) with the value specified by the null hypothesis for the population statistic (for example, a mean change in blood pressure of zero). The name of the test statistic, and how it is calculated, will depend on the test you are conducting. For example, a \(t\) test calculates a \(t\) value, a one-way ANOVA calculates an \(F\) value and a chi-square test of independence calculates a chi-square value. In each case, a large test statistic indicates a large difference between the hypothesised value and the sample statistic.
The test will also calculate a \(p\) value (or probability value). In some cases, two \(p\) values may be provided: a one-sided \(p\) value and a two-sided \(p\) value. You should use the two-sided \(p\) value unless you have previously decided to conduct a one-sided hypothesis test. The \(p\) value gives the probability of obtaining the observed test statistic, or one more extreme, if the null hypothesis is true. A small \(p\) value means that a result this extreme would be unlikely if the null hypothesis were true. It is the \(p\) value, rather than the test statistic itself, that is interpreted when deciding whether to reject the null hypothesis.
But how small does the \(p\) value need to be? To decide when to reject the null hypothesis, we need to choose a level of significance. This tells us how small the \(p\) value needs to be before we reject the null hypothesis. The level of significance is denoted by \(\alpha\) and is typically \(.05\) (\(5\%\)), although other values can also be used. Note that if:
\(p\) value \(\leqslant \alpha\) : A result this extreme, or more extreme, would have a probability of less than or equal to \(\alpha\%\) if the null hypothesis were true. Therefore, we reject the null hypothesis in favour of the alternative hypothesis. This means that the difference or relationship we have hypothesised about is statistically significant.
\(p\) value \( > \alpha\): A result this extreme, or more extreme, would have a probability greater than \(\alpha\) if the null hypothesis were true. Therefore, we cannot reject the null hypothesis. This means that the difference or relationship we have hypothesised about is not statistically significant.
If you would like to practise interpreting \(p\) values, have a go at the following activity.
At this point, it is important to note that confidence intervals can also be used to determine whether a difference or relationship is statistically significant. For example, using the sample from our blood pressure example, we can calculate a \(95\%\) confidence interval. This gives a range of plausible values for the difference in mean blood pressure in the population. If this confidence interval does not contain \(0\), then \(0\) is not considered a plausible value for the population difference. This indicates that the difference is statistically significant at the \(5\%\) level.
Confidence intervals are useful because they provide more information than statistical significance alone. They also provide information about the magnitude and direction of a difference (or relationship).
If you would like to test your understanding of this concept, have a go at this activity.
Because hypothesis testing uses incomplete information from a sample to draw conclusions about a complete population, there is always a possibility of making an error when deciding whether to reject the null hypothesis. This can happen even if our calculations are correct. There are two possible types of error:
Type I error: This occurs when we reject a null hypothesis that is actually correct. The probability of making a Type I error is equal to the level of significance \(\alpha\). This is why we generally select a low value for \(\alpha\), such as \(0.05\).
Type II error: This occurs when we do not reject a null hypothesis that is actually incorrect. The probability of making a Type II error is denoted by \(\beta\), and it is usually desirable for this to be \(0.2\) or below.
To minimise the risk of a Type II error, a power analysis is often used to determine an appropriate sample size. The power of a statistical test is the probability that the test will detect an effect if one actually exists. Power is therefore the opposite of the Type II error rate and can be expressed as \(1-\beta\). Therefore, to keep the Type II error rate \(\leq 0.2\), the power needs to be \(\geq 0.8\)
The power of a test depends on three factors:
You can use this information to calculate the power of a test using software, for example SPSS (Version 27 or above). Alternatively, and ideally, you can use software to determine an appropriate sample size needed to achieve a power of \(\geq 0.8\)
If you would like to test your understanding of the different error types, have a go at the following activity.
Statistical significance is influenced by sample size. This means that with a very large sample, very small differences may be statistically significant. With a very small sample, even large differences may not be statistically significant. For this reason, it is often useful to consider practical significance as well. This can be assessed by calculating an effect size.
The effect size provides information about whether a difference or relationship is meaningful in a practical sense (i.e. in real life). Effect size is calculated differently for different statistical tests. Details on how to calculate effect size for each test are provided on later pages of this module.
Different inferential statistical tests are used depending on the type of hypothesis being tested and the data being analysed. The following pages explain some of the most common tests. First, however, it is important to understand two main types of statistical tests:
Parametric tests: These make assumptions about the distribution of the data. They usually involve at least one continuous variable and commonly make assumptions relating to the normal distribution.
Non-parametric tests: These do not require continuous data to follow a particular distribution, such as the normal distribution. They can also be used with some types of data that are not continuous, such as ordinal data. Many non-parametric tests use the ranks or order of the data rather than the original values.
As a general rule, if the assumptions of a parametric test are met, the parametric test should be used because it generally has greater statistical power. If these assumptions are not met, or if the type of data is not suitable for a parametric test, a non-parametric test should be used instead.
The following pages explain five of the most commonly used parametric tests, with reference to their non-parametric alternatives, as well as one commonly used non-parametric test.