Type I and Type II Errors
In hypothesis testing there are two types of errors that can result in the experimenter drawing the wrong conclusion. These errors are expressed as probabilities (α and β risks), and are always present to some extent. The goal is to make sure these errors are reasonably small, given the actions that will or will not be taken as a result of the hypothesis test.
Type I Error – A conclusion that a true change has occurred in the underlying population, when in reality it has not. The probability of making a Type I error is the α risk.
Type II Error – A conclusion that the underlying population has not changed, when it reality it has. The probability of making a Type II error is the β risk.
Typical values for acceptable α and β risks are 5% and 10% respectively.