Residency · Residency · Preventive Medicine
Bayesian vs. Frequentist Approaches in Public Health
Overview
Frequentist and Bayesian statistics represent two fundamentally different philosophical approaches to statistical inference. Frequentist methods currently dominate epidemiologic practice, but Bayesian methods are gaining traction as computational barriers diminish and their advantages become more widely appreciated. Understanding both frameworks is essential for interpreting research, making sound public health decisions, and succeeding on board examinations.
Frequentist Statistics
Core Philosophy
In the frequentist framework, probability is defined as the long-run frequency of events in repeated experiments. Parameters of interest — such as the true population mean — are considered fixed but unknown quantities. The data themselves are random, meaning that if the experiment were repeated many times, different datasets would be obtained each time. Inference is based on the sampling distribution of a test statistic: how that statistic would behave across hypothetical repetitions of the study.
Key Concepts
The p-value is the probability of observing data as extreme as or more extreme than what was actually observed, assuming the null hypothesis is true. It is emphatically not the probability that the null hypothesis is true, nor the probability that results are due to chance. The conventional threshold of p < 0.05 is arbitrary, originally proposed by Fisher as a rough guide rather than a rigid cutoff. A 95% confidence interval means that if the study were repeated many times, 95% of the constructed intervals would contain the true parameter value — it does not mean there is a 95% probability that the true value lies within any particular interval. Type I error (alpha) is the probability of rejecting the null hypothesis when it is actually true (a false positive), while Type II error (beta) is the probability of failing to reject the null when it is actually false (a false negative). Power, calculated as 1 minus beta, is the probability of correctly detecting a true effect.
Hypothesis Testing Framework
The frequentist hypothesis testing framework specifies a null hypothesis (H0) of no association or no difference, and an alternative hypothesis (H1) that an association or difference exists. The null is rejected when the p-value falls below alpha, typically set at 0.05. When multiple comparisons are performed, the risk of Type I errors increases, which is sometimes addressed through corrections such as the Bonferroni method, which divides alpha by the number of tests conducted.
Limitations of Frequentist Methods
P-values are widely misinterpreted by both researchers and clinicians. The dichotomous thinking they encourage — categorizing results as "significant" or "not significant" — discourages nuanced interpretation of evidence. Because p-values depend heavily on sample size, very large studies can produce statistically significant results for clinically meaningless differences. The frequentist framework cannot directly state the probability that a given hypothesis is true. The replication crisis in science has highlighted how practices like p-hacking and publication bias exploit the vulnerabilities of frequentist threshold-based inference.
Bayesian Statistics
Core Philosophy
In the Bayesian framework, probability represents a degree of belief or uncertainty about a parameter rather than a long-run frequency. Parameters are treated as random variables possessing probability distributions. Prior knowledge is formally incorporated into the analysis, and new data update those prior beliefs to produce posterior beliefs through Bayes' theorem.
Key Concepts
The prior distribution represents existing knowledge or beliefs about a parameter before observing new data. Informative priors draw on previous studies, expert opinion, or biological knowledge. Non-informative (vague) priors express minimal prior knowledge and allow the data to dominate. Skeptical priors are centered on the null hypothesis and require strong evidence to shift the posterior away from it. The likelihood function — the probability of the observed data given a parameter value — is the same mathematical quantity used in frequentist statistics. The posterior distribution represents updated belief after combining the prior and the likelihood via Bayes' theorem, with the posterior being proportional to the prior multiplied by the likelihood. A 95% Bayesian credible interval has a direct and intuitive interpretation: there is a 95% probability that the parameter lies within this interval.
Bayes' Theorem
Bayes' theorem states that P(A|B) equals P(B|A) times P(A) divided by P(B). In clinical terms, the posterior probability equals the likelihood times the prior probability, divided by the marginal probability of the data. This principle is applied routinely in diagnostic testing: the post-test probability of disease depends on the pre-test probability (prevalence) combined with the test's characteristics (sensitivity and specificity).
Advantages of Bayesian Methods
Bayesian methods produce direct probability statements about parameters, which is what clinicians actually want to know. They formally incorporate prior knowledge, which is particularly valuable when historical data exist. They do not require multiple comparison corrections in the frequentist sense. They enable sequential analysis, updating conclusions as new data accumulate without inflating error rates. They perform better with small samples and rare events. And decision analysis — weighing costs, benefits, and uncertainties — naturally fits within a Bayesian framework.
Limitations of Bayesian Methods
The choice of prior distribution can influence results, raising concerns about subjectivity. Bayesian computations are intensive, though modern Markov Chain Monte Carlo (MCMC) methods have largely overcome this barrier. The approach remains less familiar to most clinicians and manuscript reviewers. Sensitivity analysis examining the influence of prior choice is essential for credibility. And Bayesian results can be more difficult to explain to non-statistical audiences unfamiliar with the framework.
| Feature | Frequentist | Bayesian |
|---|---|---|
| Definition of probability | Long-run frequency of events | Degree of belief/uncertainty |
| Parameters | Fixed but unknown | Random variables with distributions |
| Data | Random (varies across repetitions) | Fixed (observed) |
| Prior information | Not formally incorporated | Formally incorporated via prior distribution |
| Key output | P-value, confidence interval | Posterior distribution, credible interval |
| Interpretation of interval | 95% of intervals from repeated samples contain the true value | 95% probability the parameter lies within the interval |
| Multiple comparisons | Requires correction (e.g., Bonferroni) | Handled naturally through hierarchical models |
| Small samples | May perform poorly | Performs well with informative priors |
| Sequential analysis | Inflates Type I error without correction | Naturally accommodated through updating |
| Primary limitation | Widely misinterpreted; dichotomous thinking | Choice of prior can be subjective |
Clinical Applications
Diagnostic Testing
The reasoning clinicians perform every day when interpreting diagnostic tests is inherently Bayesian. Starting with a pre-test probability (influenced by prevalence and clinical features), combining it with likelihood ratios derived from test characteristics, and arriving at a post-test probability represents a direct application of Bayes' theorem — even when clinicians simply call it "clinical judgment."
Drug and Vaccine Development
Bayesian adaptive trials allow modification of treatment arms, sample sizes, and randomization ratios based on accumulating data during the trial. The FDA has accepted Bayesian designs for medical device trials. Bayesian borrowing of historical control data can reduce the sample sizes needed for new studies while maintaining appropriate uncertainty quantification.
Public Health Surveillance
Bayesian methods improve disease mapping by borrowing strength across geographic regions through spatial smoothing. Small-area estimation — producing reliable disease rate estimates for areas with few cases — benefits substantially from Bayesian hierarchical models. Real-time outbreak detection leverages sequential Bayesian updating to identify emerging signals.
Meta-Analysis
Bayesian meta-analysis allows incorporation of prior information and produces full posterior distributions for effect sizes rather than point estimates with confidence intervals. It handles between-study heterogeneity more naturally through hierarchical models and accommodates uncertainty in all parameters simultaneously.
The P-Value Debate and Modern Proposals
ASA Statement on P-Values (2016)
The American Statistical Association issued a landmark statement clarifying that p-values do not measure the probability that a hypothesis is true, do not measure the size or clinical importance of an effect, and should not serve as the sole basis for scientific conclusions. The statement emphasized that proper inference requires full reporting of effect sizes, confidence intervals, and study design details.
Proposed Alternatives and Reforms
Multiple reform proposals have emerged. Benjamin and colleagues (2018) proposed lowering the threshold for "statistical significance" to p < 0.005. Amrhein and colleagues (2019), in a Nature editorial signed by over 800 scientists, called for abandoning the term "statistically significant" entirely. Routine reporting of effect sizes and confidence intervals alongside or instead of p-values has gained support. Using Bayesian methods to directly quantify evidence for and against hypotheses, and reporting Bayes factors (the ratio of evidence for one hypothesis versus another), represent additional proposed reforms.
<image>A side-by-side visual comparison of frequentist and Bayesian inference. Left panel shows the frequentist approach: a sampling distribution centered on the null, a shaded rejection region, and the p-value area marked. Right panel shows the Bayesian approach: a prior distribution curve, a likelihood function curve, and a resulting posterior distribution curve, with the 95% credible interval shaded. Annotations highlight key differences -- the frequentist panel notes "parameter is fixed, data vary" while the Bayesian panel notes "parameter varies, beliefs are updated." Professional statistical education illustration style.</image>
<image>A flowchart showing how Bayes' theorem is applied in clinical diagnostic testing. Starting with pre-test probability (prevalence), applying a positive test result with known sensitivity and specificity, calculating the positive likelihood ratio, and arriving at the post-test probability. A parallel path shows the calculation for a negative test result. The flowchart includes numerical example values and a Fagan nomogram-style graphic connecting pre-test probability, likelihood ratio, and post-test probability on three vertical axes connected by a straight line. Medical education diagram style.</image>
<image>A timeline infographic showing key milestones in the p-value and statistical inference debate. Milestones include: Fisher's introduction of p < 0.05 (1925), Neyman-Pearson hypothesis testing framework (1933), the emergence of Bayesian methods in clinical trials (1990s-2000s), the ASA statement on p-values (2016), the proposal to lower significance threshold to 0.005 (2017), and the Nature call to abandon statistical significance (2019). Each milestone includes a brief annotation about its impact on epidemiologic practice. Horizontal timeline with color-coded eras.</image>
Clinical Pearls
A p-value of 0.05 does not mean there is a 5% chance the null hypothesis is true — this remains the single most common statistical misinterpretation in medicine. A 95% confidence interval does not mean there is a 95% probability the true value lies within it; that statement actually describes a Bayesian credible interval. Bayesian thinking is already embedded in everyday clinical practice through the process of updating disease probability after diagnostic testing. Large sample sizes can produce statistically significant results for trivially small effects, so the magnitude of the effect must always be examined independently. When reading papers, focus on effect sizes and confidence intervals rather than p-values alone. Bayesian adaptive trial designs are increasingly used in vaccine and medical device development. For board examinations, know the correct interpretation of p-values and confidence intervals, and understand the application of Bayes' theorem to diagnostic testing.
References
- Wasserstein RL, Lazar NA. The ASA statement on p-values. Am Stat. 2016;70(2):129-133.
- Benjamin DJ, et al. Redefine statistical significance. Nat Hum Behav. 2018;2:6-10.
- Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567:305-307.
- Greenland S, et al. Statistical tests, P values, confidence intervals, and power. Eur J Epidemiol. 2016;31:337-350.
- Berry DA. Bayesian clinical trials. Nat Rev Drug Discov. 2006;5:27-36.
- Goodman SN. Toward evidence-based medical statistics. Ann Intern Med. 1999;130:995-1013.
