[latex]\newcommand{\pr}[1]{P(#1)} \newcommand{\var}[1]{\mbox{var}(#1)} \newcommand{\mean}[1]{\mbox{E}(#1)} \newcommand{\sd}[1]{\mbox{sd}(#1)} \newcommand{\Binomial}[3]{#1 \sim \mbox{Binomial}(#2,#3)} \newcommand{\Student}[2]{#1 \sim \mbox{Student}(#2)} \newcommand{\Normal}[3]{#1 \sim \mbox{Normal}(#2,#3)} \newcommand{\Poisson}[2]{#1 \sim \mbox{Poisson}(#2)} \newcommand{\IQR}{\mbox{IQR}}[/latex]

Normal Density Curves

In Chapter 8 we looked at two continuous random variables that might model female height, one with a uniform density curve and one with a triangular density curve. These were both simple shapes and so it was relatively easy to calculate probabilities from areas under the curves. However, the most famous density curve has a much more complicated shape, given by the probability density function
\[ f(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{1}{2} \left( \frac{x – \mu}{\sigma} \right)^2}, \]
where [latex]\mu[/latex] and [latex]\sigma[/latex] are parameters of the function.
This curve is called the Normal density curve. There is actually a family of Normal density curves since you get a different function for each pair of values of the parameters [latex]\mu[/latex] and [latex]\sigma[/latex]. The Normal curve is symmetric about the value [latex]x = \mu[/latex], as you might see from the fact that it involves [latex](x - \mu)^2[/latex]. To compare with the uniform and triangular models, the figure below shows a Normal model for female heights where [latex]\mu = 167[/latex] and [latex]\sigma = 6.6[/latex].

If [latex]X[/latex] is a continuous random variable whose probabilities are given by a Normal density curve with parameters [latex]\mu[/latex] and [latex]\sigma[/latex] then we say that [latex]X[/latex] has a Normal distribution and write [latex]\Normal{X}{\mu}{\sigma}[/latex].

Note that the name is a bit misleading since a random variable having a Normal distribution is not really ‘normal’ in the usual sense of the word. We will use the uppercase “Normal” to emphasise the difference. Another name for the function is the Gaussian density curve and so you will sometimes see this used to avoid confusion.

Although the density function for the Normal distribution is complicated, it is possible to use the formulas from Chapter 10 to show that the expected value (or mean) of the Normal distribution is [latex]\mu[/latex] and the standard deviation is [latex]\sigma[/latex] (which is why we use these symbols).

Unfortunately there is no simple formula for working out areas under the Normal density curve. Instead areas can be found using the Normal distribution table or with computer software. However, as a rough rule

• the area within 1 standard deviation of the mean is 68%,
• the area within 2 standard deviations of the mean is 95%
• the area within 3 standard deviations of the mean is 99.7%

This is sometimes called the 68-95-99.7 rule, for obvious reasons. We will make particular use of the 95% part.

Standard Units

There are a whole family of Normal distributions but it would be impractical to have a table of areas for each one. Instead we focus on one Normal distribution, with mean 0 and standard deviation 1, the standard Normal distribution.

Suppose female heights are Normally distributed with mean 167 cm and standard deviation 6.6 cm, and consider a female whose height is 180 cm. The idea of standard units is to describe how many standard deviations a value is from the mean. Here 180 is 13 cm above the mean, which is 13/6.6 = 1.97 standard deviations. Note that the units cancel, so 1.97 is dimensionless; its units are just “standard deviations”. Subtracting and dividing by constants doesn’t change the shape of the distribution, so the 1.97 is still from a Normal distribution but with mean 0 and standard deviation 1.

The historical advantage of doing this is that we only need one table of Normal probabilities. To find an area we first standardise our value and then consult the standard Normal table. The standardised value is called a z score. For example, the [latex]z[/latex] score for a female’s height of 180 cm in the Normal(167, 6.6) distribution is 1.97, as above. The probability that a random standardised value, [latex]Z[/latex], is 1.97 or greater is [latex]\pr{Z \ge 1.97} = 0.024[/latex], from the Normal distribution table. Thus 2.4% of females are taller than a female who is 180 cm tall.

In general, standardising takes a random variable [latex]\Normal{X}{\mu}{\sigma}[/latex] and gives
\[ Z = \frac{X – \mu}{\sigma} \]
so that [latex]\Normal{Z}{0}{1}[/latex].

Such calculations are not so important these days since software and graphics calculators can work out these probabilities from scratch, without having to standardise. Just like the logarithm tables of thirty years ago, standard Normal tables will one day disappear.

However, it is still useful to think in terms of standard units. For example, if male heights are Normally distributed with mean 179 cm and standard deviation 7.6 cm, then a male who is 194 cm tall is also 1.97 standard deviations above the mean. Even though 180 and 194 are quite different, they are the same relative to their distributions.

How many standard units will put you in the middle 95% of the distribution? To have 95% in the middle would mean there is 5% left over. Since the Normal distribution is symmetric this gives 2.5% in each tail. Hunting around in the Normal distribution table we find that 1.96 standard units has an area of 0.025 to the right. Thus 95% of observations will occur within 1.96 standard deviations of the mean. This is where the rough “2 standard deviations” in the 68-95-99.7 rule comes from.

Normal Probability Plots

The standard Normal density curve can be viewed as a theoretical distribution and so we can work out properties of the distribution. For instance, the median is 0, the same as the mean since the distribution is symmetric. The third quantile is the [latex]z[/latex] value with 25% of the area above it. Looking through the Normal distribution table we see this is about 0.67. In fact, [latex]Q_3 = 0.674[/latex]. Then [latex]Q_1 = -0.674[/latex] because of the symmetry about 0.

This gives us a simple way of assessing whether observed values of a continuous variable could be described by a Normal density curve. We can work out our sample values of the median, [latex]Q_1[/latex] and [latex]Q_3[/latex], as well as more quantiles if we wish, and compare them to the theoretical values. This can be done visually by making a plot where the vertical axis is our variable and the horizontal axis is for the Normal distribution, and each point compares a quantile from the data with a theoretical quantile. Such a plot is called a Normal probability plot or a Normal quantile plot.

The figure above shows a Normal probability plot for the pulse rate observations in the survey data. The horizontal axis is given in standard deviations away from the mean. The median pulse rate was 68 bpm and so on the plot there is a point at (0, 68). Similarly the first quartile was 62 bpm and so there is a point at (-0.674, 62). All the points lie roughly along a straight line which indicates a consistent correspondence between the quantiles of the data and the quantiles of the Normal distribution. This suggests the Normal distribution is a useful model of the data and does so more objectively than looking at a histogram. The horizontal stacking of points arises from the somewhat discrete nature of the pulse rate measurements.

The figure above shows a Normal probability plot for the weight observations in the survey data. This time the points do not seem to follow a straight line. The few points to the right have higher weight values than they should for a Normal density curve which suggests that the right tail of the data is more stretched out than it should be. At the same time the points to the left are not as low as they should be for a Normal density which suggests that the left tail of the data does not extend as far as it should if it was Normal. If you sketch a density curve using this description you will see you have a distribution which is skewed to the right. Normal probability plots are thus also useful for detecting skewness in data.

In later chapters we will use methods which are based on the assumption that data is close to Normal. Probability plots are a useful way of assessing how valid this assumption is for a particular set of observations.

The 1.5 [latex]\times[/latex] IQR Rule Revisited

In Chapter 4 we introduced the “1.5 [latex]\times[/latex] IQR rule” for flagging possible outliers in data, typically used in making box plots. But why is this an appropriate rule and where does the “1.5” come from?

Suppose our population does in fact have a perfect Normal distribution and we make a sample from it without error. What is the probability that a genuine observation will be flagged as an outlier by the rule?

We noted in the previous section that [latex]Q_1 = -0.674[/latex] and [latex]Q_3 = 0.674[/latex] so [latex]\IQR = Q_3 - Q_1 = 1.348[/latex]. Thus we would reject any value with a [latex]z[/latex] score of
\[ 0.674 + 1.5 \times 1.348 = 2.696 \]
or higher. The probability of a real value being this high is just [latex]\pr{Z \ge 2.696} = 0.003[/latex] from the Normal distribution table. The chance of a genuine observation being rejected on the lower side is also 0.003, so together the chance of being flagged as an outlier is about 0.006. This is then the probability of making a mistake when using this rule with Normal data and is not too bad, less than 1%.

However, this again illustrates why you should always investigate observations flagged as outliers before thinking of removing them from your analysis. With a sample of 100 observations in which there really are no “outliers”, with a 1% error rate we’d expect to see an observation or two flagged. This error rate is much higher for skewed distributions, so you should be particularly cautious then.

Histograms

For large numbers of observations, histograms and other density estimates are often very useful in assessing symmetry, the most important aspect of Normality to test. However, be careful with histograms from small data sets. For example, the figure below shows ten histograms from samples of size 20.

None of these look as nice as the idealised Normal density in the previous figure but in fact all of them show samples from a mathematical process that generates data from a perfect Normal distribution. In particular, each of these histograms is showing data from a symmetric distribution, so this is a chance to get a feel for how far from symmetric a histogram should look before you declare it “skewed”.

Random Numbers

To conclude this section, the table below gives 540 random numbers taken from the standard Normal distribution, with mean 0 and standard deviation 1. These can be used for making your own data sets for practice exercises. If you want to generate random data from a variable [latex]\Normal{X}{\mu}{\sigma}[/latex] then note that the formula we have used for standardising can be rearranged to give
\[ X = \mu + \sigma Z, \]
where [latex]\Normal{Z}{0}{1}[/latex].

Random numbers from Normal(0,1)

For example, suppose you want to make some data that could have come from a population of female heights with the Normal(167,6.6) distribution. The first number given in the table above is -0.96 so we would obtain a height of
\[ 167 + 6.6\times(-0.96) = 167 – 6.34 = 160.66, \]
so the first female would have a height of 160.7 cm. The second number, 0.46, gives a height of 170.0 cm, and so on. You could continue this process for all the 10 numbers on the first line, giving 10 random heights. You could then try visualising these heights, or analyse them with one of the methods later in the book. The advantage is that you know what population they have come from and so you can compare your answers based on the random data with the truth about the population.

Standard Normal distribution

This table gives [latex]P(Z \ge z)[/latex] for [latex]\Normal{Z}{0}{1}[/latex]. Critical values of the Normal distribution, the [latex]z^{*}[/latex] values such that [latex]\pr{Z \ge z^{*}} = p[/latex] for a particular [latex]p[/latex] can be found from the [latex]\infty[/latex] row of the table of critical values of Student’s T distribution.