If you find any typos, errors, or places where the text may be improved, please let me know by adding an annotation using hypothes.is. To add an annotation, select some text and then click the on the pop-up menu. To see the annotations of others, click the in the upper right-hand corner of the page.

C Review on hypothesis testing

The process of hypothesis testing has an interesting analogy with a trial that helps on understanding the elements present in a formal hypothesis test in an intuitive way.

Hypothesis testing Trial
Null hypothesis \(H_0\) Accused of comitting a crime. It has the “presumption of innocence,” which means that it is not guilty until there is enough evidence to supporting its guilt
Sample \(X_1,\ldots,X_n\) Collection of small evidences supporting innocence and guilt. These evidences contain a certain degree of uncontrollable randomness because of how they were collected and the context regarding the case
Statistic \(T_n\) Summary of the evicences presented by the prosecutor and defense lawyer
Distribution of \(T_n\) under \(H_0\) The judge conducting the trial. Evaluates the evidence presented by both sides and presents a verdict for \(H_0\)
Significance level \(\alpha\) \(1-\alpha\) is the strength of evidences required by the judge for condemning \(H_0\). The judge allows evidences that on average condemn \(100\alpha\%\) of the innocents, due to the randomness inherent to the evidence collection process. \(\alpha=0.05\) is considered a reasonable level
\(p\)-value Decision of the judge that measures the degree of compatibility, in a scale \(0\)\(1\), of the presumption of innocence with the summary of the evidences presented. If \(p\)-value\(<\alpha\), \(H_0\) is declared guilty. Otherwise, is declared not guilty
\(H_0\) is rejected \(H_0\) is declared guilty: there are strong evidences supporting its guilt
\(H_0\) is not rejected \(H_0\) is declared not guilty: either is innocent or there are no enough evidences supporting its guilt

More formally, the \(p\)-value of an hypothesis test about \(H0\) is defined as:

The \(p\)-value is the probability of obtaining a statistic more unfavourable to \(H_0\) than the observed, assuming that \(H_0\) is true.

Therefore, if the \(p\)-value is small (smaller than the chosen level \(\alpha\)), it is unlikely that the evidence against \(H_0\) is due to randomness. As a consequence, \(H_0\) is rejected. If the \(p\)-value is large (larger than \(\alpha\)), then it is more possible that the evidences against \(H_0\) are merely due to the randomness of the data. In this case, we do not reject \(H_0\).

If \(H_0\) holds, then the \(p\)-value (which is a random variable) is distributed uniformly in \((0,1)\). If \(H_0\) does not hold, then the distribution of the \(p\)-value is not uniform but concentrated at \(0\) (where the rejections of \(H_0\) take place).