Introduction to statistics

Many of the statistical tests covered later in this module assume that continuous data follows, or approximately follows, a normal distribution, or that the sample size is large enough for the sampling distribution of the mean to be approximately normal. But what is a normal distribution? What is a sampling distribution? How large does a sample need to be? And how can you tell whether a continuous variable is approximately normally distributed?

This page answers these questions and provides important background knowledge for the inferential statistics topics covered in the following pages.

The normal distribution

The normal distribution is a special type of distribution that many naturally occurring continuous variables approximately follow. Because of this, many inferential statistical tests, known as parametric tests, are based on the properties of the normal distribution. These properties are:

• The mean, median and mode are equal.
• Fixed proportions of the data lie within certain numbers of standard deviations from the mean. In particular, approximately 68% of the data lies within one standard deviation of the mean, 95% lies within two standard deviations, and 99.7% lies within three standard deviations.

These properties give the normal distribution its characteristic bell-shaped curve, shown below. Note that the percentages displayed in the graph are given to two decimal places, while the percentages above are rounded values. Also note that the symbols \(\mu\) and \(\sigma\) represent the population mean and population standard deviation respectively.

If you would like to practise interpreting data from a normal distribution, have a go at the following activity.

Sampling distributions

The descriptive statistics covered in the previous page of this module are used to analyse the data you have access to, which in most cases is a random sample of a larger population. It is important to remember though that any random sample is only one of many possible samples that could be selected from a population, and that sample statistics such as the sample mean will likely vary from sample to sample. For example, we could measure the heights of a random sample of 100 Curtin students and calculate the mean height, then we could select a different random sample of 100 Curtin students and calculate another mean height, which would likely be different, and so on. If we continued this process, we would obtain a mean for each possible sample of the same size.

All of these sample statistics (such as the mean heights in our example) form a distribution of their own, known as a sampling distribution. If we had access to all of these values then we could display the sampling distribution using a histogram, just as we would for any other set of data. In practice, however, this is not possible because it would require collecting data from every possible sample. Instead, inferential statistics use data from a single random sample to estimate properties of the sampling distribution.

Sampling distributions are therefore an important concept in statistics. In particular, it is useful to understand the following properties of the sampling distribution of the mean:

• As the sample size increases, the shape of the sampling distribution of the mean becomes increasingly similar to a normal distribution. This important result is known as the Central Limit Theorem. Note that a sample size of 30 or more is often considered large enough for the sampling distribution to be approximately normal, and therefore to fulfil any normality requirements, although a larger sample size may be needed if the distribution is highly skewed.
• The mean of the sampling distribution is equal to the population mean.
• The standard deviation of the sampling distribution is equal to the population standard deviation divided by the square root of the sample size. This value is known as the standard error of the mean.

Testing for normality

If the inferential statistical test you want to use requires a normally distributed continuous variable (or variables) and your sample size is not sufficiently large (typically 30 or above), then you will need to test for normality. Unfortunately, there is no single test that can determine with certainty whether data are normally distributed. Instead, several statistics and graphs can be used to assess whether the data are approximately normal. Data are rarely perfectly normally distributed, and some deviation from a normal distribution is often acceptable, so examining multiple indicators provides a more complete understanding of the data and helps you make a more informed judgement about normality.

The following statistics and graphs can be used to assess normality:

• Mean, median and mode
  For a perfectly normal distribution, the mean, median and mode are all equal. Comparing these values is therefore a simple way to begin assessing normality. In practice, small differences are common, particularly for the mode, and this is usually less concerning than differences between the mean and median.

• Skewness
  Skewness measures the symmetry of a distribution. A skewness value of \(0\) indicates a perfectly symmetrical distribution, while negative values indicate negative skew (a longer tail on the left) and positive values indicate positive skew (a longer tail on the right). If the data is approximately normally distributed then the skewness should be close to \(0\), and as a general guide, values between \(-1\) and \(1\) are usually considered acceptable. You can also calculate a z-score for skewness by dividing it by its standard error and this z-score should fall between \(-1.96\) and \(1.96\)

• Kurtosis
  Kurtosis measures the shape of a distribution’s tails. A normal distribution has a kurtosis of \(3\), although many researchers and software programs (such as SPSS) report excess kurtosis instead. Excess kurtosis is calculated by subtracting \(3\) from the kurtosis, giving the normal distribution a value of \(0\). Using excess kurtosis, positive values indicate heavier tails and negative values indicate lighter tails, but again a value between \(−1\) and \(1\) is generally considered acceptable. You can also calculate a z-score for excess kurtosis by dividing it by its standard error. This z-score should fall between \(-1.96\) and \(1.96\)

• Normality test (e.g. Shapiro-Wilk)
  The Shapiro-Wilk test is a statistical test of normality, which tests the null hypothesis that the data comes from a normally distributed population. Another commonly used normality test is the Kolmogorov-Smirnov test. A significance (\(p\)) value greater than \(0.05\) suggests that there is insufficient evidence to conclude that the data is not normally distributed, which provides support for assuming normality (more information about hypothesis testing is provided in the Inferential statistics page of this module). For larger samples, normality tests can become very sensitive and may identify minor departures from normality that are not practically important. For this reason, they are usually interpreted alongside the other statistics and graphs described on this page.

• Histogram
  If the data is approximately normally distributed the histogram should be roughly symmetrical and centred around the mean (Figure 1). If there is a longer tail on the left the distribution is negatively skewed (Figure 2), and if there is a longer tail on the right the distribution is positively skewed (Figure 3).

• Stem and leaf plot
  A stem and leaf plot displays the frequency of values in a data set using ‘stems’ and ‘leaves’. For example, Figure 4 below shows that there is one value of \(63\), two values of \(65\), six values of either \(66\) or \(67\), and so on. A stem and leaf plot can be interpreted in much the same way as a histogram, only rotated on its side.

• Normal Q-Q plot
  If the data is approximately normally distributed, the points on a normal Q-Q plot will lie close to the straight diagonal line (Figure 7). If the data is not normally distributed, the points will tend to deviate from this line (Figures 8 and 9). Another version of this graph, called a detrended Q-Q plot, is sometimes also used. In a detrended Q-Q plot, there should be a roughly equal number of points above and below the horizontal line, with no obvious pattern.

• Boxplot

If the data is approximately normally distributed, the median should be positioned near the centre of the box and the whiskers should be of similar length. Ideally, there should be few or no outliers (Figure 10). Alternatively, the boxplot may show negative skew (Figure 11) or positive skew (Figure 12).

After examining these indicators of normality, you should draw an overall conclusion based on the evidence as a whole. For example, you might conclude that the data is approximately normally distributed, or that it shows positive or negative skew.

For a worked example of assessing normality, you make like to view the Introduction to SPSS or Introduction to Stata modules. You can also practise assessing whether data approximates a normal distribution by completing the following activity.

Transforming variables

If the tests for normality indicate that a variable is not approximately normally distributed, and the sample size is not sufficiently large, it may be possible to transform the variable so that its distribution more closely resembles a normal distribution. For example, if the data is negatively skewed, taking the square of the values may help reduce the skewness. If the data is positively skewed, applying a natural logarithm (\(ln\)), square root or reciprocal transformation may be useful.

After the data has been transformed, it should be tested for normality again to see whether the transformation has improved the distribution. If the transformed data is now approximately normally distributed, any further statistical analysis should be carried out using the transformed data. If the distribution is still not approximately normal, a non-parametric test should be used instead.