[latex]\newcommand{\pr}[1]{P(#1)} \newcommand{\var}[1]{\mbox{var}(#1)} \newcommand{\mean}[1]{\mbox{E}(#1)} \newcommand{\sd}[1]{\mbox{sd}(#1)} \newcommand{\Binomial}[3]{#1 \sim \mbox{Binomial}(#2,#3)} \newcommand{\Student}[2]{#1 \sim \mbox{Student}(#2)} \newcommand{\Normal}[3]{#1 \sim \mbox{Normal}(#2,#3)} \newcommand{\Poisson}[2]{#1 \sim \mbox{Poisson}(#2)} \newcommand{\se}[1]{\mbox{se}(#1)} \newcommand{\prbig}[1]{P\left(#1\right)} \newcommand{\degc}{$^{\circ}$C}[/latex]

Testing Randomness

In contrast to the algorithmic random digits seen in Chapter 2, the table following this chapter gives 1800 digits from a human asked to create a random sequence. Are these digits random?

We will look at the first half of this sequence, the first 900 digits, leaving an analysis of the second half as an exercise. There are actually many different criteria for a sequence being “random” in this context, one of which is that the outcomes should all be equally likely. Here we would expect each digit to appear 900/10 = 90 times. The observed values are given in the table below with a bar chart of these shown in the following figure. This is a one-way table, giving observed counts for a single categorical variable.

Observed and expected counts for the first 900 digits

There are certainly deviations from what we would expect, ranging from only 40 occurrences for ‘9’ up to 145 for ‘4’. Even if the digits were truly random we would not expect to get exactly 90 of each one appearing. But are the observed deviations plausible if they were truly random? This is a standard hypothesis test setting. We want to know the probability of getting values as far away (or further) than those observed by chance if they really were equally likely.

This is a good chance to reflect on the basic ideas of hypothesis testing. We are not estimating a parameter here, and so will not be talking about confidence intervals. Instead we write [latex]H_0[/latex] in words as

[latex]H_0[/latex]: observations follow hypothesised distribution.

The alternative is the very general statement

[latex]H_1[/latex]: observations do not follow hypothesised distribution.

This is known as a goodness-of-fit test and we need some way of measuring how close the observed counts are to the expected counts. An obvious measure is to add up all the differences between the observed and expected counts, since we would expect this to be bigger if there were bigger deviations. However this sum is always 0 because the positive and negative differences always cancel out. (Why?) We could fix this by adding up the absolute differences, but as usual we add up the squared differences, just as we did for the sample standard deviation. This gives the statistic
\[ \sum (\mbox{observed} – \mbox{expected})^2, \]
where the sum is over all the categories (the 10 digits).
This, however, is not perfect as it does not take into account the relative size of deviations. For example, an observed value of 20 would be the same distance from an expected value of 10 as an observed value of 1010 would be from an expected value of 1000. However the first is much more significant since the observation was double the expected, while the second is not much of a difference at all. To capture this we take the ratio of the squared difference by the expected value, giving
\[ \chi^2 = \sum \frac{(\mbox{observed} – \mbox{expected})^2}{\mbox{expected}}. \]
Here [latex]\chi[/latex] is the Greek letter chi, and this statistic is called the chi-square statistic. If there is evidence against the null hypothesis then we would expect [latex]\chi^2[/latex] to be large. Here we find
\[ \chi^2 = \frac{(47 – 90)^2}{90} + \frac{(101 – 90)^2}{90} + \cdots + \frac{(40 – 90)^2}{90} = 132.07. \]

How do we know if 132.07 could simply be due to sampling variability? We need to know the sampling distribution of this statistic, assuming that [latex]H_0[/latex] is true. This distribution is called the chi-square distribution. Like the [latex]t[/latex] distribution, there is a different chi-square distribution for each number of categories. Here we have 10 categories but the sum of the differences between observed and expected is always 0, so there are only 9 free differences in the analysis. As before, we call this the degrees of freedom of the chi-square statistic.

The figure below shows the [latex]\chi^2_9[/latex] distribution, the chi-square distribution with 9 degrees of freedom, along with the [latex]\chi^2_1[/latex], [latex]\chi^2_4[/latex] and [latex]\chi^2_8[/latex] distributions for comparison. Since we are squaring everything the value of [latex]\chi^2[/latex] can never be negative but there is no real limit on how big [latex]\chi^2[/latex] can be, so this is a rather skewed distribution.

Like the other continuous distributions we have seen, there is no simple way of working out areas under the [latex]\chi^2_9[/latex] density curve. The table below gives the areas under the [latex]\chi^2_1[/latex] distribution as an example but it is impractical to provide tables for each degrees of freedom and is also worthless since computer packages can provide these areas easily.

[latex]\chi^2(1)[/latex] distribution

This table gives [latex]\pr{X^2 \ge x}[/latex] where [latex]X^2 \sim \chi^2_1[/latex].

The following table provides the critical values for a range of degrees of freedom so you can see their general pattern. Unlike the [latex]t[/latex] distributions, the critical values here keep getting higher as the degrees of freedom increase, not surprising since [latex]\chi^2[/latex] is the sum of more and more terms.

[latex]\chi^2[/latex] distribution

This table gives [latex]x^{*}[/latex] such that [latex]\pr{X^2 \ge x^{*}} = p[/latex], where [latex]X^2 \sim \chi^2(\mbox{df})[/latex].

The [latex]P[/latex]-value we want is [latex]\pr{X^2 \ge 132.07}[/latex], where [latex]X^2 \sim \chi^2_9[/latex]. For 9 degrees of freedom in the table above this [latex]P[/latex]-value is far below 0.001. Note that this is a two-sided [latex]P[/latex]-value already, since positive and negative deviations have been squared and combined, so there is no need to multiply it by 2. This is very strong evidence against [latex]H_0[/latex] so in conclusion there is very strong evidence to suggest that the human-generated numbers are not uniformly distributed.

Assumptions

Note that the observed counts in our data are discrete and so the [latex]\chi^2[/latex] statistic is also discrete, even though it might look like a continuous decimal number. However, the [latex]\chi^2[/latex] distribution we are using for hypothesis testing is continuous, so the underlying assumption is that this continuous approximation to the real discrete distribution is a good one. This is analogous to using the Normal distribution to approximate the Binomial distribution for proportion tests.

To satisfy this assumption we use the rule of thumb that all expected counts should be at least 1 and 80% of them should be at least 5. In the random digits example this is justified, with all expected counts equal to 90. In the section below we’ll see an example where we need to combine groups to satisfy the assumption.

Correlation Test of Randomness

Before we continue discussing categorical data, note that there are other requirements that a genuinely random sequence of numbers should satisfy. One important one is that consecutive numbers should be independent. This is particularly important if we were using a random number generator on a calculator or computer to help choose random samples for an experiment. All of the statistical tests we have described assume that samples are independent of each other and so a poor random number generator could undermine our studies.

One way of testing for independence here is to tally occurrences of the 100 possible pairs of digits. If the outcomes were independent then these 100 pairs should be equally likely and we could use a chi-square statistic with 99 degrees of freedom to test this uniformity.

Another method is to make a scatter plot where each point represents a digit and the digit that followed it in the sequence. Such a plot is shown in the following figure, where jittering has been used to separate points which would otherwise be obscured (since this is discrete data). If one digit and the next were independent then there should be no association present in this plot. However, there are noticeable gaps (an ‘8’ was never followed by a ‘1’) and very dense combinations (‘6’ was often followed by ‘5’). This leads to a smoothed line suggesting that perhaps there is a positive association between one digit and the next.

We can support the visual impression by calculating the correlation coefficient, [latex]r = +0.232[/latex]. This is not particularly large but it is significantly different from 0 ([latex]p \lt 0.001[/latex]). This gives strong evidence that low digits tend to be followed by low digits and high digits tend to be followed by high digits. Thus there is further evidence that the digits produced are not genuinely random.

Parametric Distributions

Relationship to Proportion Test

Two-Way Tables

In Chapter 17 we looked at the effect of nicotine inhalers on smoking reduction using a comparison between two proportions. We can try to determine whether the inhalers are beneficial by testing for an association in the two-way table of counts. The following table shows this data again, with marginal totals included. Our null hypothesis is that the inhaler contents and the reduction outcome are independent, while the alternative hypothesis is simply that they are not.

Sustained reductions after 4 months of inhaler use

From the marginal distributions we see that 200/400 = 0.5 of the subjects had the nicotine inhaler, while 70/400 = 0.175 of the subjects had a reduction. If there was no association between inhaler contents and reduction then these outcomes should be independent of each other. We should then be able to multiply their proportions together to estimate the proportion of subjects having nicotine and having a reduction,
\[ 0.5 \times 0.175 = 0.0875. \]
Thus we would expect 8.75% of all subjects to have this combination. Now 8.75% of 400 is 35, compared to the observed value of 52. Is this a significant difference? We can use a chi-square test to find out.

Firstly, we calculate the other three expected counts. Note that we divide by 400 twice in getting our proportions but then multiply by it at the end. We can save one step and give the simple formula
\[ \mbox{expected count } = \frac{\mbox{row total } \times \mbox{ column total}}{\mbox{total}}. \]
For example, we would expect the count of subjects in the placebo group who don’t sustain a reduction to be
\[ \frac{200 \times 330}{400} = 165. \]
The table below gives all the expected counts.

Expected counts for inhaler data

We can simply work out the chi-square statistic as before,
\[ \chi^2 = \frac{(52 – 35)^2}{35} + \cdots + \frac{(182 – 165)^2}{165} = 20.02. \]
This statistic has a [latex]\chi^2_{\mbox{df}}[/latex] distribution with
\[ \mbox{df } = (\mbox{rows } – 1) \times (\mbox{columns } – 1), \]
since the degrees of freedom from the two variables multiply in the same way that you multiply the number of rows and columns to find the number of cells. For this example, df = 1. The [latex]\chi^2(1)[/latex] table or the [latex]\chi^2[/latex] table tell us the [latex]P[/latex]-value is very close to 0. Thus there is very strong evidence of an association between the type of inhaler and whether a reduction was sustained.

Note that [latex]\sqrt{20.02} = 4.47[/latex], the [latex]z[/latex] statistic we found in Chapter 17 with the pooled sample proportion, so this test for association is identical to the two-sample proportion test using the Normal approximation. The advantage of the chi-square test is that it can be applied to tables with more than two rows or columns.

Fisher’s Exact Test

For small sample sizes the test based on the [latex]\chi^2[/latex] distribution will usually give a poor approximation to the [latex]P[/latex]-values. An alternative is to use Fisher’s exact test (Glantz, 2002). This procedure enumerates all the possible tables that would be as unusual as the one obtained assuming no association or simulates the generation of such tables if the number of possibilities is too big. Either method is straightforward but can require a lot of calculation and so is usually left to a computer.

Simpson’s Paradox

Appleton et al. (1996) surveyed women twenty years after they were part of a study in 1972-1974 on thyroid and heart disease. The following table shows the survival status at the time of the second survey of the 1314 women who had been classified as either a current smoker or as never having smoked in the original survey.

Two-way table of smoking and survival

Of the 582 women who smoked, 443 were still alive after twenty years, a survival rate of 76%. Of the 732 women who didn’t smoke, 502 were still alive, a survival rate of 69%. That is interesting: it seems that, for the population these women came from, smoking might actual help survival. A chi-square test of association gives a [latex]P[/latex]-value of 0.003, strong evidence that smoking status and survival are related.

This seems like good news for smokers! But now consider the table below which shows the same data but as a three-way table with an extra variable for age group.

Three-way table of age group, smoking and survival

We can now look at the relationship between smoking status and survival for the different age groups.

• For 18-44 year olds, 269 out of 288 smokers survived (93%) compared to 327 out of 340 nonsmokers (96%). For this age group it was better to be a nonsmoker.
• For 45-64 year olds, 167 out of 245 smokers survived (68%) compared to 147 out of 199 nonsmokers (74%). For this age group it was also better to be a nonsmoker.
• For women 65 and over, 7 out of 49 smokers survived (14%) compared to 28 out of 193 nonsmokers (15%). For this age group it was also slightly better to be a nonsmoker.

So for each age group the survival rate was higher for nonsmokers. That seems odd since above we saw that altogether it was smokers who had the better survival rate. The same data are giving us completely opposite conclusions!

This phenomenon is known as Simpson’s paradox. We will leave you to figure out which is the correct conclusion and why the other one gives the opposite result.

Human random digits

These digits were generated by a human asked to type a sequence of 1800 random digits.