# Lecture 20: Simple Linear Regression
## Statistics / Biostatistics
---
## Learning Objectives
By the end of this lecture, students will be able to:
1. Describe the simple linear regression model and its assumptions
2. Estimate regression coefficients using the least squares method
3. Interpret the slope, intercept, and R-squared
4. Perform hypothesis tests and construct confidence intervals for regression parameters
5. Use residual analysis to assess model fit
---
## Lecture Content
### I. The Simple Linear Regression Model
Simple linear regression models the relationship between one continuous predictor (X) and one continuous outcome (Y). The model equation is Y = beta_0 + beta_1 * X + epsilon, where beta_0 is the y-intercept (the value of Y when X = 0), beta_1 is the slope (the change in Y for a one-unit increase in X), and epsilon is the random error term, assumed to follow N(0, sigma^2). The goal is to find the "best-fitting" straight line through the data.
Regression differs from correlation in an important way. Correlation measures association and is symmetric, with no implied directionality. Regression models a directional predictive relationship where X predicts Y, making it inherently asymmetric.
### II. Least Squares Estimation
The method of least squares minimizes the sum of squared residuals: minimize sum of (y_i - y-hat_i)^2 = sum of (y_i - b_0 - b_1 * x_i)^2. The estimated slope is b_1 = sum[(x_i - x-bar)(y_i - y-bar)] / sum[(x_i - x-bar)^2] = S_xy / S_xx, and the estimated intercept is b_0 = y-bar - b_1 * x-bar. The fitted (predicted) values are y-hat_i = b_0 + b_1 * x_i, and the residuals are e_i = y_i - y-hat_i (observed minus predicted). An important geometric property is that the regression line always passes through the point (x-bar, y-bar).
A scatterplot with a fitted regression line. Data points (x_i, y_i) are shown as dots. The regression line y-hat = b_0 + b_1*X is drawn through the data. Vertical dashed lines from several data points to the regression line represent residuals (e_i). The intercept b_0 is labeled where the line crosses the y-axis. The slope b_1 is illustrated with a "rise over run" triangle on the line. The point (x-bar, y-bar) is marked with a distinct symbol on the line.
### III. Interpretation of Coefficients
The **slope (b_1)** is interpreted as: "For each one-unit increase in X, Y changes by b_1 units on average." For example, for each additional year of age, systolic BP might increase by 0.5 mmHg. The sign of the slope indicates direction -- positive means a direct relationship, negative means an inverse one.
The **intercept (b_0)** is the predicted value of Y when X = 0. It may or may not be clinically meaningful, depending on whether X = 0 falls within the data range. If X is age and the intercept predicts systolic BP at age 0, this extrapolation is meaningless. More broadly, extrapolation should be avoided: do not use the model to predict Y for X values far outside the observed range.
### IV. R-Squared (Coefficient of Determination)
R^2 = SS_Regression / SS_Total = 1 - SS_Residual / SS_Total. It represents the proportion of total variability in Y that is explained by the linear relationship with X. Its range is 0 to 1. In simple linear regression, R^2 equals the square of the Pearson correlation (r^2). For example, R^2 = 0.64 means 64% of the variation in Y is accounted for by X.
A high R^2 does not guarantee the model is correct -- residuals should always be checked. Conversely, a low R^2 does not mean the relationship is unimportant; clinical relevance must be assessed independently of the proportion of variance explained.
### V. Inference for Regression Coefficients
The standard error of the slope is SE(b_1) = sqrt[MS_Residual / sum(x_i - x-bar)^2]. The hypothesis test for the slope tests H0: beta_1 = 0 (no linear relationship), with t = b_1 / SE(b_1), and df = n - 2. This test is equivalent to testing whether the correlation is zero. The confidence interval for the slope is b_1 +/- t_(alpha/2, n-2) * SE(b_1). The overall regression can also be tested with an F-test: F = MS_Regression / MS_Residual, with df1 = 1 and df2 = n-2. In simple linear regression, this F-test is equivalent to the t-test for the slope, since F = t^2.
### VI. Assumptions of Simple Linear Regression (LINE)
The assumptions of simple linear regression can be remembered by the acronym LINE. **L**inearity means the relationship between X and Y is linear; this is checked by examining the scatterplot and the residuals versus fitted values plot, which should show no pattern. **I**ndependence means observations are independent of each other; this is checked through study design, and for time-ordered data, autocorrelation should be assessed. **N**ormality means the residuals are normally distributed; this is checked with a histogram or Q-Q plot of residuals. **E**qual variance (homoscedasticity) means the variance of residuals is constant across all values of X; the residuals versus fitted values plot should show constant spread with no funnel shape.
A four-panel residual diagnostic plot. Panel A (Residuals vs. Fitted): A random scatter of points around zero with constant spread -- assumptions met. Panel B (Residuals vs. Fitted): A funnel shape (variance increasing with fitted values) -- heteroscedasticity. Panel C (Residuals vs. Fitted): A curved pattern -- nonlinearity. Panel D (Q-Q plot of residuals): Points closely following the diagonal -- normality of residuals confirmed. Each panel is annotated with the diagnosis and recommended remedial action.
### VII. Prediction
Two types of intervals can be constructed at a given value X = x_0. A **confidence interval for the mean response** estimates E(Y | X = x_0) with uncertainty about the mean; this produces a narrower interval. A **prediction interval for a new individual observation** predicts a single future Y value and is wider because it includes both uncertainty about the mean and individual variability. Both intervals are narrowest at X = x-bar and widen as x_0 moves away from the center of the data.
### VIII. Clinical Example
Consider the research question: is there a linear relationship between BMI (X) and fasting blood glucose (Y)? With n = 50 patients, the analysis yields b_0 = 50.2 mg/dL, b_1 = 2.8 mg/dL per BMI unit, SE(b_1) = 0.65, t = 2.8/0.65 = 4.31, df = 48, p < 0.001, and R^2 = 0.28. The interpretation is that for each 1-unit increase in BMI, fasting glucose increases by 2.8 mg/dL on average. The relationship is statistically significant (p < 0.001), and BMI explains 28% of the variability in fasting glucose. The remaining 72% is due to other factors such as genetics, diet, and medication.
A complete regression analysis output figure. Panel A: Scatterplot of BMI vs. fasting glucose with the regression line, 95% confidence band (for the mean), and 95% prediction band (wider). Panel B: The residual vs. fitted values plot showing adequate assumptions. Panel C: A summary table with the regression equation, coefficient estimates, standard errors, t-statistics, p-values, and R-squared. The figure is labeled as a clinical example of BMI predicting fasting blood glucose.
---