The Logic of Hypothesis Testing
We want to know the answer to a research question. We determine our null and alternative hypotheses. Now it is time to make a decision.
The decision is either going to be…
- reject the null hypothesis or…
- fail to reject the null hypothesis.
Consider the following table. The table shows the decision/conclusion of the hypothesis test and the unknown “reality”, or truth. We do not know if the null is true or if it is false. If the null is false and we reject it, then we made the correct decision. If the null hypothesis is true and we fail to reject it, then we made the correct decision.
| Decision | Reality | |
|---|---|---|
| H0 is true | H0 is false | |
| Reject H0, (conclude Ha) | Correct decision | |
| Fail to reject H0 | Correct decision | |
So what happens when we do not make the correct decision?
When doing hypothesis testing, two types of mistakes may be made and we call them Type I error and Type II error. If we reject the null hypothesis when it is true, then we made a type I error. If the null hypothesis is false and we failed to reject it, we made another error called a Type II error.
| Decision | Reality | |
|---|---|---|
| H0 is true | H0 is false | |
| Reject H0, (conclude Ha) | Type I error | Correct decision |
| Fail to reject H0 | Correct decision | Type II error |
Types of errors
- Type I error
- When we reject the null hypothesis when the null hypothesis is true.
- Type II error
- When we fail to reject the null hypothesis when the null hypothesis is false.
The “reality”, or truth, about the null hypothesis is unknown and therefore we do not know if we have made the correct decision or if we committed an error. We can, however, define the likelihood of these events.
- α (‘Alpha’)
- The probability of committing a Type I error. Also known as the significance level.
- β (‘Beta’)
- The probability of committing a Type II error.
- Power
- Power is the probability the null hypothesis is rejected given that it is false (ie. 1−β)
α and β are probabilities of committing an error so we want these values to be low. However, we cannot decrease both. As α decreases, β increases.
Example 6-1 Cont’d…
A man, Mr. Orangejuice, goes to trial and is tried for the murder of his ex-wife. He is either guilty or not guilty. We found before that…
- H0: Mr. Orangejuice is innocent
- Ha: Mr. Orangejuice is guilty
Interpret Type I error, α, Type II error, β.
- Type I Error:
- Type I error is committed if we reject H0 when it is true. In other words, when the man is innocent but found guilty.
- α:
- α is the probability of a Type I error, or in other words, it is the probability that Mr. Orangejuice is innocent but found guilty.
- Type II Error:
- Type II error is committed if we fail to reject H0 when it is false. In other words, when the man is guilty but found not guilty.
- β:
- β is the probability of a Type II error, or in other words, it is the probability that Mr. Orangejuice is guilty but found not guilty.
As you can see here, the Type I error (putting an innocent man in jail) is the more serious error. Ethically, it is more serious to put an innocent man in jail than to let a guilty man go free. So to minimize the probability of a type I error we would choose a smaller significance level.
