[latex]\newcommand{\pr}[1]{P(#1)} \newcommand{\var}[1]{\mbox{var}(#1)} \newcommand{\mean}[1]{\mbox{E}(#1)} \newcommand{\sd}[1]{\mbox{sd}(#1)} \newcommand{\Binomial}[3]{#1 \sim \mbox{Binomial}(#2,#3)} \newcommand{\Student}[2]{#1 \sim \mbox{Student}(#2)} \newcommand{\Normal}[3]{#1 \sim \mbox{Normal}(#2,#3)} \newcommand{\Poisson}[2]{#1 \sim \mbox{Poisson}(#2)} \newcommand{\se}[1]{\mbox{se}(#1)}[/latex]

Consider the following sequence of reasoning based on what we have seen so far.

• The sample mean has roughly a Normal distribution with mean [latex]\mu[/latex] and standard deviation [latex]\sigma/\sqrt{n}[/latex].
• In a Normal distribution about 95% of observations occur within 1.96 standard deviations of the mean.
• So in 95% of samples the sample mean will be within [latex]1.96\sigma/\sqrt{n}[/latex] of [latex]\mu[/latex].
• Reversing this, in 95% of samples [latex]\mu[/latex] will be within [latex]1.96\sigma/\sqrt{n}[/latex] of [latex]\overline{x}[/latex].

This tells us that when we use the sample mean to estimate the population mean, we can also give an idea of how far away the population mean could be from our estimate. We say we are 95% confident that the population mean is
\[ \overline{x} \pm 1.96\frac{\sigma}{\sqrt{n}}, \]
or that it is in the interval
\[ \left(\overline{x} – 1.96\frac{\sigma}{\sqrt{n}}, \overline{x} + 1.96\frac{\sigma}{\sqrt{n}} \right). \]
We call this range a confidence interval for the population mean.

This is great, allowing us to say something concrete about a population based on our sample. Unfortunately this expression for a confidence interval is also useless in practice, since there is typically no reason why we would actually know [latex]\sigma[/latex], the population standard deviation. Before we can calculate confidence intervals from real data we need to deal with this issue.

Standard Error

In Chapter 13 we saw that if [latex]X[/latex] is a random variable with mean [latex]\mu[/latex] and standard deviation [latex]\sigma[/latex] then
\[ Z = \frac{\overline{X} – \mu}{\sigma/\sqrt{n}}, \]
has (approximately) the standard Normal distribution. The standard deviation of the sample mean, [latex]\sigma/\sqrt{n}[/latex], allows us to quantify the precision of the sample mean as an estimate of the population mean [latex]\mu[/latex], as in the confidence interval above.

So how can we calculate the precision of the sample mean when we don’t know [latex]\sigma[/latex]? The simple solution is to estimate [latex]\sigma[/latex] by the sample standard deviation, [latex]s[/latex]. This gives an estimated standard deviation of the sample mean,
\[ \se{\overline{x}} = \frac{s}{\sqrt{n}}. \]
The estimated standard deviation of a statistic is its standard error.

Note that the standard error of the sample mean, [latex]\se{\overline{x}}[/latex], will be different for each sample, since [latex]s[/latex] will be different for each sample. We have used a lowercase [latex]\overline{x}[/latex] to emphasise that it only makes sense to talk about the standard error of a particular sample mean, [latex]\overline{x}[/latex], rather than of the sample mean process, [latex]\overline{X}[/latex].

Student’s t Distribution

Using the standard error to standardise gives a statistic
\[ \frac{\overline{X} – \mu}{S/\sqrt{n}}, \]
where [latex]\overline{X}[/latex] and [latex]S[/latex] are the sample mean and sample standard deviation. [latex]S[/latex] is a random variable, since it is different for each sample, and so this statistic will have more variability than just the variability from the sampling distribution of [latex]\overline{X}[/latex]. Does it still have a Normal distribution?

To check this we made 10000 samples of 5 adult males from Arcadia, recording the mean height in each sample. Here we actually know the entire population, the 1273 adult males in the town, and so we can calculate the population mean [latex]\mu[/latex] = 178.3 cm and standard deviation [latex]\sigma[/latex] = 7.00 cm. The figure below shows a density plot of the standardised [latex]z[/latex] values with the standard Normal distribution overlaid on it. The agreement is pretty close — most values are between -3 and 3, as expected for a Normal distribution.

Now instead of standardising by the known standard deviation, let us use the sample standard deviations, different for each sample. The distribution is shown in the figure below, again with the standard Normal distribution overlaid.

The difference is somewhat subtle but important. There is extra variability in the standardised values, ranging from about -10 to 10 instead of -3 to 3. We say the distribution has “fatter” tails than the standard Normal since the density is heavier in the tails. This is more clearly illustrated in the following figure, where the `S’ shape of the Normal probability plot suggests the distribution is stretched out in both directions compared to a Normal density.

The sample standard deviation is a random quantity and has added some extra uncertainty to our standardised values. We need to compensate for this when working with sample means and their standard errors. For example, a confidence interval based on 1.96 standard deviations will not give 95% confidence anymore; it may only produce a correct interval 91% of the time. We will need to go out extra standard deviations to maintain the same level of confidence.

If [latex]\overline{X}[/latex] and [latex]S[/latex] are the sample mean and sample standard deviation from a random sample then the random variable
\[ T = \frac{\overline{X} – \mu}{S/\sqrt{n}} \]
has the distribution known as Student’s t distribution. This distribution was developed by William Gosset, a chemist at the Guinness Brewery in Dublin from 1899. He wasn’t a statistician by training but found lots of small data sets to deal with at the Brewery. Over the years he developed methods for working with this data, and published details of the [latex]t[/latex] distribution in 1908 under the pen-name “Student” (Student, 1908).

There are actually many [latex]t[/latex] distributions. To see why, consider the distribution in the following figure. This shows the results of a similar simulation to before but with samples of size 25 instead of size 5.

The distribution is now closer to Normal so the [latex]t[/latex] distribution with [latex]n = 25[/latex] appears to be different to the [latex]t[/latex] distribution with [latex]n = 5[/latex] in our earlier figure. With samples of only 5 people the sample standard deviation [latex]s[/latex] is a pretty unreliable estimate of the unknown [latex]\sigma[/latex], but for samples of 25 people it is more accurate and so the [latex]t[/latex] value calculation is closer to the original [latex]z[/latex] calculation.

In fact we will have a different [latex]t[/latex] distribution for every sample size. What is important is the amount of information we have about the population variability, the quantity we called degrees of freedom in Chapter 5. We call the distribution of [latex]T[/latex] from samples of size 5 the [latex]t[/latex] distribution with 4 degrees of freedom, written [latex]t(4)[/latex]. In general, the [latex]t[/latex] values calculated from samples of size [latex]n[/latex] will have the [latex]t(n-1)[/latex] distribution.

T(4) Distribution

This table gives [latex]\pr{T \ge t}[/latex] for [latex]\Student{T}{4}[/latex]

The table above gives probabilities for the [latex]t(4)[/latex] distribution, with a similar structure to the table for the standard Normal distribution. However, since the distribution depends on the degrees of freedom, in practice we would need a whole book of such tables to cater for different sample sizes. A table such as the one below is used instead. This just gives the critical values for the distribution, the [latex]t[/latex] scores that have a certain probability to their right. This is exactly what is needed for forming confidence intervals and can also be used to decide significance at certain levels for hypothesis tests, as we will see in Chapter 15.

For large samples there is essentially no difference between the [latex]t[/latex] distribution and the Normal distribution, as you can see from the table below. You may see books referring to using the [latex]t[/latex] distribution as “small sample” methods and using the Normal distribution as “large sample” methods. In fact, whenever you use a sample standard deviation you use a [latex]t[/latex] distribution. It is just that for large samples it does not really matter.

Critical values of Student's T distribution

This table gives [latex]t^{*}[/latex] such that [latex]\pr{T \ge t^{*}} = p[/latex], where [latex]\Student{T}{\mbox{df}}[/latex].

Confidence Interval for a Mean

We can now calculate a confidence interval for a population mean [latex]\mu[/latex], using the [latex]t[/latex] distribution in place of the Normal distribution. The general formula is
\[ \overline{x} \pm t_{n-1}^{*} \frac{s}{\sqrt{n}}, \]
where [latex]t_{n-1}^{*}[/latex] is the number of standard errors required for the desired confidence in the [latex]t(n-1)[/latex] distribution. We’ll see how this comes together in an example.

Caffeine

Consider the increases in pulse rate of the 10 subjects in Chapter 2 who drank the caffeinated cola. These have mean [latex]\overline{x}[/latex] = 15.80 bpm with standard deviation [latex]s = 8.324[/latex] bpm. We estimate the mean increase in pulse rate for all people in this population with 15.80 bpm. The standard error of this estimate is
\[ \se{\overline{x}} = \frac{8.324}{\sqrt{10}} = 2.632 \mbox{ bpm}, \]
with [latex]10 - 1 = 9[/latex] degrees of freedom.

For a Normal distribution, 95% of outcomes occur within 1.96 standard deviations of the mean. For the [latex]t[/latex] distributions, the table of Student’s T distribution gives the [latex]t[/latex] scores that have certain areas to the right. The figure below shows that for 95% confidence we require 5% area in the tails.

The [latex]t[/latex] distributions are symmetric and so this means we need 2[latex]\frac{1}{2}[/latex]% area to the right. Thus we look under the [latex]p = 0.025[/latex] column of the table of Student’s T distribution. With 9 degrees of freedom the critical value is 2.262 standard errors. Note that this is higher than 1.96, reflecting the fact that we need to have a wider interval to account for the uncertainty that comes from not knowing [latex]\sigma[/latex].

We can now assemble our confidence interval. Based on the sample of 10 subjects we are 95% confident that the mean increase in pulse rate from drinking 250 mL of caffeinated diet cola is
\begin{eqnarray*}
15.80 \pm 2.262 \frac{8.324}{\sqrt{10}} & = &15.80 \pm 2.262 \times 2.632 \\
& = & 15.80 \pm 5.95 \mbox{ bpm}.
\end{eqnarray*}
Thus we are 95% confident that the mean increase is between about 9.8 bpm and 21.8 bpm.

Reflect on where this interval comes from. In particular, note that the 95% is saying that an interval generated in this way will contain the true mean 95% of the time. A confidence interval is a random interval and the confidence level gives the probability of this interval containing the value of interest.

The margin of error for this interval was 5.95 bpm. This can be used as a measure of the precision of the estimate. Confidence intervals can be written either as the actual interval, such as (9.8, 21.8) or 9.8 — 21.8, or as the estimate and margin of error, such as
\[ 15.80 \pm 5.95. \]
If you are interested in seeing whether a particular value for the mean is plausible, then the interval is more useful. If you don’t care about particular values but just want to make clear how accurate your estimate is, then the second form is better.

From the calculation of margin of error, you can see that the estimate can be made more precise by increasing the sample size [latex]n[/latex]. We can also improve precision by accepting lower confidence levels. For example, if we were happy with 90% confidence then the table of Student’s T distribution shows that 1.833 standard errors would be fine. The following figure shows the spectrum of confidence intervals for the mean increase in pulse rate across a range of confidence levels. The three levels most commonly used, 90%, 95% and 99%, are indicated by vertical lines.

Choosing Sample Size

The margin of error for the increase in pulse rate was 5.95 bpm. Suppose we wanted to be able to estimate the mean increase in pulse rate with a margin of error of 2 bpm. If we keep the 95% confidence level, our only choice is to alter sample sizes.

In an ideal world where we knew the population standard deviation, [latex]\sigma[/latex], the margin of error is
\[ m = 1.96\frac{\sigma}{\sqrt{n}}. \]
Rearranging this gives
\[ n = \left(\frac{1.96\sigma}{m}\right)^2, \]
a simple formula for calculating the required sample size. In practice we need an estimate of [latex]\sigma[/latex] in order to use this formula. For the caffeine example we can treat the data observed as coming from a pilot study. This is one reason for pilot studies, to give initial estimates of variability in order to help plan the main study for meeting certain objectives. Here our pilot study estimates [latex]\sigma[/latex] with [latex]s[/latex] = 8.324 and we would like our main study to estimate the mean to within [latex]m[/latex] = 2 bpm at 95% confidence. Thus we require
\[ n = \left(\frac{1.96 s}{m}\right)^2 = \left(\frac{1.96 \times 8.324}{2.0}\right)^2 = 66.5, \]
so at least 67 subjects are needed in a trial to obtain the desired accuracy.

Given that the [latex]z^* = 1.96[/latex] and [latex]s = 8.324[/latex] are fixed in this calculation, the sample size is proportional to the square of the reciprocal of the desired margin of error,
\[ n \propto \left(\frac{1}{m}\right)^2. \]
Thus in general if you want to halve the margin of error you will need to increase the sample size by a factor of four. The figure below shows this relationship in terms of the ratio of the desired margin of error to the standard deviation. For example, wanting a margin of error of 2 bpm when our estimated standard deviation was 8.324 bpm gives a ratio of [latex]2/8.324 = 0.24[/latex]. The height of the plot at 0.24 is around 67, the required sample size.

Note that the 1.96 in the above calculations could also be replaced by the appropriate [latex]t^*[/latex], though this is hard since we don’t know what [latex]n-1[/latex] will be. An iterative method can be used but in this setting we are usually planning large studies for which the difference between [latex]t^*[/latex] and 1.96 will be negligible. (For [latex]n = 67[/latex], [latex]t^*[/latex] is less than 2.00.) The amount of variability in [latex]s[/latex] between the pilot study and the main study will almost certainly overshadow this small difference in the number of standard errors.

Prediction Intervals

A confidence interval gives a range of plausible values for the mean of a population. For example, the [latex]n[/latex] = 10 subjects who drank caffeinated cola had a mean increase in pulse rate of [latex]\overline{x}[/latex] = 15.80 bpm and standard deviation [latex]s[/latex] = 8.324 bpm. The 95% confidence interval for the mean increase in pulse rate was
\[ 15.80 \pm 5.95 \mbox{ bpm}. \]
Again this means we are 95% confident that the mean increase in pulse rate is between 9.8 bpm and 21.8 bpm.

Suppose instead we want to predict the increase in pulse rate experienced by another person who drinks 250 mL of caffeinated cola. As always, there are two questions we need to ask and you should take the time to think about the answers before reading on:

1. What value should we use for our prediction?
2. What is the precision of this prediction likely to be?

If we want to “predict” a population mean then we have the answers to these. We use the sample mean as our prediction and can use its standard error, in conjunction with the [latex]t[/latex] distribution, to quantify its accuracy. In this case we usually say we are “estimating” a population mean; we will use “predict” to refer to making predictions about individual outcomes.

What value should we use to predict the increase for a new person from this population? If we don’t have any other information then our best bet is to use the sample mean, 15.80 bpm. One reason for doing this is that the sample mean is the mode of the Normal density curve we might use to describe increases in pulse rate and so is the most likely outcome.

So our prediction for an individual increase is simply the sample mean, 15.80 bpm. To measure how precise this prediction is we need to take into account two sources of variability. Firstly, there is the variability in the sample mean as an estimate. This is what the standard error measures, with a value in this case of 2.632 bpm, and is what we have used in our confidence intervals. To predict an individual outcome we must also add the natural variability in the population. This is estimated by the sample standard deviation, 8.324 bpm. For our prediction interval error we combine these to obtain
\[ \sqrt{2.632^2 + 8.324^2} = 8.730 \mbox{ bpm}. \]
We use [latex]t_{9}^{*}[/latex] as before, so a 95% prediction interval is
\[ 15.80 \pm t_{9}^{*} \; \times 8.730 = 15.80 \pm 19.75 \mbox{ bpm}. \]
This is quite a wide interval, which is not surprising since wanting to be 95% sure of the increase in pulse rate is going to include most possible increases.

A general formula for a prediction interval is
\[ \overline{x} \pm t_{n-1}^{*} s \sqrt{1 + \frac{1}{n}}. \]
Such intervals are rarely used in this context. We will return to them when talking about predictions from least-squares fits, a more common use, in Chapter 18.

Assumptions for t Methods

Recall that the [latex]t[/latex] distribution arose from a Normal distribution where we didn’t know the standard deviation. If the population we are sampling from has a Normal distribution then the sample mean has a Normal distribution too. Thus we can use the [latex]t[/latex] methods and all the probabilities will be correct. That is, 95% of the time we make a 95% confidence interval it will contain the population mean.

What if the population is not Normal? In that case the Central Limit Theorem says that the sample mean is approximately Normal and that the approximation gets better as the sample size increases. As long as this approximation is okay then it is fine to use the [latex]t[/latex] methods. Wild and Seber (2000) give the following recommendations as to how big [latex]n[/latex] should be in order for the approximation to be reasonable.

• For small samples (roughly [latex]n \lt 15[/latex]) only use the [latex]t[/latex] methods if the data are close to symmetric and there are no outliers.
• For moderate samples ([latex]15 \le n \le 40[/latex]) use the [latex]t[/latex] methods as long as there are no outliers or strong skewness in the data.
• For large samples (roughly [latex]n \gt 40[/latex]) use the [latex]t[/latex] methods even in the presence of strong skewness, though outliers may still affect results.

Normal probability plots (see Chapter 12) are a useful means for checking for skewness and outliers.

If we cannot use the [latex]t[/latex] procedures then there are various options. One is to use nonparametric methods, as we will discuss in Chapter 24. These make different assumptions about the data and so can work where the [latex]t[/latex] methods are not appropriate.

Transforming Data

The other general approach is to transform the data in some way to help satisfy the [latex]t[/latex] assumptions. For example, taking logarithms can make a skewed distribution more symmetric. Analysis can be carried out on the log values and then conclusions can be exponentiated to return to the original units.

Maximum Drug Concentration

In Chapter 6 we gave a time plot of drug concentrations from a single subject in a bioequivalence study. The maximum drug concentration during the 8 hours for this subject was 627 [latex]\mu[/latex]g/L for the reference formulation and 704 [latex]\mu[/latex]g/L for the test formulation. The maximum concentrations for the remaining 23 subjects are shown in the following table, along with the time taken to reach that concentration. One condition for bioequivalence is that the maximum concentration of the test formulation should be between 80% and 125% of the maximum concentration of the reference formulation.

Maximum drug concentration ([asciimath]\mu[/asciimath]g/L) and time to maximum (hours)

	Max Concentration		Time to Max
Subject	Test	Reference	Test	Reference
1	704	627	1.33	4.00
2	767	618	0.98	1.02
3	530	336	0.98	2.00
4	1322	1369	1.67	1.35
5	796	984	4.48	1.33
6	576	423	0.98	1.00
7	3097	2212	1.33	2.00
8	1429	1696	1.33	2.33
9	626	690	2.67	1.33
10	433	430	1.33	1.33
11	4136	3088	1.00	2.02
12	527	458	2.00	1.33
13	1223	1516	3.00	2.00
14	1267	674	1.67	2.02
15	374	603	1.65	2.00
16	1026	926	1.33	1.67
17	2265	2062	1.67	1.32
18	445	461	1.32	1.35
19	774	690	1.33	4.00
20	796	984	4.48	1.33
21	1226	1499	3.00	2.00
22	553	481	2.00	1.33
23	1284	1592	3.00	2.00
24	1330	655	1.67	1.67

The figure below shows a comparison of the maximum drug concentrations between the test and reference groups.

For a new formulation to be bioequivalent to a reference formulation, one of the conditions is that the maximum concentration from the new formulation is between 80% and 125% that of the reference formulation. We need a confidence interval to estimate the difference between maximum concentrations and then we can see if this holds in this study.

However, the previous figure suggests that the distributions of maximum concentrations are highly skewed, undermining the applicability of a [latex]t[/latex] confidence interval. It is standard to take logarithms in this case to help make the data more symmetric — the figure below shows the same data plotted on a log scale. The transformed data are not perfectly symmetric, but they are a lot closer and, given the sample size, a [latex]t[/latex] procedure is now appropriate.

The second step is to notice that these are not independent sets of measurements since they come from the same patients. These two figures are actually quite misleading by suggesting a comparison between two groups. What we are interested in is whether there is a difference between the maximum concentrations in each patient, so the simple approach is to take the difference between each pair of transformed measurements. These values are shown in the table below for you to check.

Differences in log-transformed maximum drug concentrations

0.050	0.094	0.198	-0.015	-0.092	0.134
0.146	-0.074	-0.042	0.003	0.127	0.061
-0.093	0.274	-0.207	0.045	0.041	-0.015
0.050	-0.092	-0.087	0.061	-0.093	0.308

This is now a one-sample problem. The sample mean of the [latex]n = 24[/latex] observations is 0.0326 with standard deviation 0.12448. This gives a 95% confidence interval (for the population difference between the transformed maximum concentrations) of
\[ 0.0326 \pm t_{23}^{*} \frac{0.12448}{\sqrt{24}} = 0.0326 \pm 0.0525, \]
a range of -0.0199 to 0.0851. This is hard to interpret directly, but recall that
\[ \log(\mbox{Test}) – \log(\mbox{Reference}) = \log \left(\frac{\mbox{Test}}{\mbox{Reference}}\right), \]
so undoing our transformation will give information about the ratio of the maximum concentrations. In fact, the values in the previous table could have been calculated by taking the logs of the ratios, rather than the differences in the logs. For example, the first subject had a maximum concentration of 704 [latex]\mu[/latex]g/L for the test formulation compared to 627 [latex]\mu[/latex]g/L for the reference formulation. This gives a ratio of [latex]704/627 = 1.123[/latex], and then [latex]\log(1.123) = 0.050[/latex], as before.

Thus the estimated mean ratio is
\[ 10^{0.0326} = 1.078, \]
so we estimate that the test formulation is giving about a 7.8% higher maximum concentration on average than the reference formulation. We can do the same for the confidence interval bounds, giving a 95% interval for the true ratio to be
\[ (10^{-0.0199}, 10^{0.0851}) = (0.955, 1.216). \]
Note that the middle of this range is 1.086, which is different to the estimated mean, 1.078. All the confidence intervals we have seen so far have been symmetric about their estimate, but when you use a transformation like this you may end up with an asymmetric interval.

The upper range of this interval gives a ratio with the test maximum 21.6% higher than the reference maximum. This is below the prescribed limit of 125%, so this suggests that the two formulations are bioequivalent in terms of maximum drug concentration.

Geometric Mean

Suppose we have [latex]n[/latex] observations [latex]x_1, x_2, \ldots, x_n[/latex] and we work out the sample mean of their log-transformed values,
\begin{eqnarray*}
\frac{\log(x_1) + \log(x_2) + \cdots + \log(x_n)}{n} & = & \frac{\log(x_1 \times x_2 \times \cdots \times x_n)}{n} \\
& = & \log( \sqrt[n]{x_1 \times x_2 \times \cdots \times x_n})
\end{eqnarray*}
In the above example we did this calculation for [latex]n=24[/latex] observations and found the sample mean of 0.0326 for transformed ratios. However, we wanted an estimate of the mean ratio and so we removed the logarithm by calculating [latex]10^{0.0326} = 1.078[/latex]. Doing this for the general formula gives
\[ \sqrt[n]{x_1 \times x_2 \times \cdots \times x_n}. \]
You may have seen this formula before. It is the geometric mean of the [latex]n[/latex] values. Thus the mean ratio we calculate using the log-transformed ratios is actually the geometric mean, not the regular sample mean (also known as the arithmetic mean).

This highlights one of the roles of the geometric mean. It is always less than the sample mean and so for right-skewed data it may be a more appropriate measure of the centre of the distribution. Indeed that is why we were using the log-transformed data, overcoming the strong skewness present in the data.