Premed · Premed · Statistics Biostatistics
Lecture 14: Hypothesis Testing - Framework and Logic
Statistics / Biostatistics
Learning Objectives
By the end of this lecture, students will be able to:
- State null and alternative hypotheses for a given research question
- Describe the logic of hypothesis testing (test statistic, p-value, decision)
- Distinguish between one-tailed and two-tailed tests
- Interpret p-values correctly and avoid common misconceptions
- Explain the relationship between hypothesis tests and confidence intervals
Lecture Content
I. The Logic of Hypothesis Testing
Hypothesis testing is a formal framework for making decisions about population parameters based on sample data. It is analogous to a legal trial: the defendant (null hypothesis) is presumed innocent until proven guilty, evidence (data) is collected and evaluated, and a verdict is reached based on whether the evidence is strong enough. The process follows six steps: state the null and alternative hypotheses, choose a significance level (alpha), collect data and compute the test statistic, determine the p-value, make a decision to reject or fail to reject the null hypothesis, and interpret the result in context.
II. Null and Alternative Hypotheses
The null hypothesis (H0) is a statement of "no effect" or "no difference." It represents the status quo or the skeptical position. Examples include H0: mu = 120 (the mean systolic BP equals 120) or H0: p1 = p2 (two proportions are equal). The alternative hypothesis (H1 or Ha) is what the researcher wants to show -- the claim or suspicion. Examples include H1: mu does not equal 120 or H1: p1 does not equal p2 (two-sided), or H1: mu > 120 (one-sided).
A critical point is that the null hypothesis is never "accepted." We either reject it or fail to reject it. "Fail to reject" means the data do not provide sufficient evidence against H0 -- it does not mean H0 is true.
III. One-Tailed vs. Two-Tailed Tests
A two-tailed (two-sided) test uses H1: mu does not equal mu_0 and rejects H0 if the test statistic is extreme in either direction. It is more conservative and generally preferred unless there is strong a priori justification for a directional hypothesis. A one-tailed (one-sided) test uses H1: mu > mu_0 or H1: mu < mu_0 and rejects H0 only in one direction. It is more powerful for detecting an effect in the specified direction but must be specified before data collection -- it cannot be chosen after seeing the results. In clinical research, two-sided tests are the standard unless regulatory guidelines specify otherwise.
<image>Three normal distribution curves illustrating rejection regions. Panel A (Two-tailed): The curve has both tails shaded (alpha/2 = 0.025 each) beyond z = +/-1.96, with the center unshaded region labeled "Fail to reject H0." Panel B (Right-tailed): Only the right tail is shaded (alpha = 0.05) beyond z = 1.645. Panel C (Left-tailed): Only the left tail is shaded beyond z = -1.645. Each panel shows the critical value(s) and the direction of the alternative hypothesis.</image>
IV. The Test Statistic
A test statistic is a standardized value calculated from sample data that measures how far the sample estimate is from the null hypothesis value. Its general form is: test statistic = (estimate - null value) / standard error. For a single mean with sigma unknown, t = (x-bar - mu_0) / (s / sqrt(n)). For a single proportion, z = (p-hat - p_0) / sqrt[p_0(1-p_0)/n]. Larger absolute values of the test statistic provide stronger evidence against H0.
V. The P-Value
The p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one calculated, assuming H0 is true. It measures the strength of evidence against H0: a small p-value indicates strong evidence against H0 because the observed result would be unlikely if H0 were true. The decision rule is to reject H0 if the p-value is less than or equal to alpha, and to fail to reject if the p-value exceeds alpha. The common significance level is alpha = 0.05.
It is essential to understand what the p-value is NOT. It is not the probability that H0 is true, not the probability that the result occurred by chance, and not the probability of making an error. The p-value IS the probability of the observed (or more extreme) data, given that H0 is true.
VI. Significance Level and Critical Values
The significance level (alpha) is the pre-specified threshold for rejecting H0 and represents the maximum acceptable probability of a Type I error (rejecting H0 when it is true). It is commonly set at 0.05, though 0.01 or 0.10 may be used depending on context.
An alternative to computing p-values is the critical value approach, where the test statistic is compared directly to a critical value. For a two-sided z-test at alpha = 0.05, reject if |z| > 1.96. For a two-sided t-test, reject if |t| > t_(alpha/2, df). Both the p-value approach and the critical value approach always yield the same decision.
<image>A flowchart of the hypothesis testing procedure. Box 1: "State H0 and H1." Box 2: "Choose alpha (e.g., 0.05)." Box 3: "Collect data, compute test statistic." Box 4: "Calculate p-value." Decision diamond: "Is p-value <= alpha?" Yes arrow leads to "Reject H0 -- result is statistically significant." No arrow leads to "Fail to reject H0 -- result is not statistically significant." A final box reminds: "Interpret in clinical context."</image>
VII. Statistical Significance vs. Clinical Significance
Statistical significance means the p-value is less than or equal to alpha, indicating that the observed effect is unlikely due to chance alone. Clinical significance means the effect is large enough to matter in clinical practice. These are not the same thing. A very large study can detect a tiny, clinically meaningless difference and report it as statistically significant. Conversely, a small study may fail to detect a clinically meaningful difference because it lacks sufficient statistical power.
For this reason, it is essential to always examine the effect size and confidence interval, not just the p-value. For example, a new drug that lowers systolic blood pressure by only 1 mmHg with p = 0.01 is statistically significant but clinically trivial.
VIII. Relationship Between Hypothesis Tests and Confidence Intervals
A two-sided hypothesis test at level alpha and a (1-alpha)*100% confidence interval are complementary. If the CI excludes the null value, the test rejects H0 at level alpha. If the CI includes the null value, the test fails to reject H0. Confidence intervals provide more information than hypothesis tests because they show the range of plausible values for the parameter and convey both the direction and magnitude of the effect. Modern guidelines recommend reporting both CIs and p-values for the most complete picture of the results.
<image>A number line illustrating the CI-hypothesis test duality. Two scenarios are shown. Scenario A: The 95% CI for the mean difference (2.5 to 8.1) does not include 0 -- "Reject H0, p < 0.05." Scenario B: The 95% CI for the mean difference (-1.3 to 5.7) includes 0 -- "Fail to reject H0, p > 0.05." In both scenarios, the point estimate and CI are displayed with the null value (0) marked by a vertical line.</image>
