# Lecture 11: Sampling Distributions and the Central Limit Theorem

## Statistics / Biostatistics

---

## Learning Objectives

By the end of this lecture, students will be able to:

1. Define a sampling distribution and explain its role in inference
2. Describe the sampling distribution of the sample mean
3. State the Central Limit Theorem and its conditions
4. Calculate probabilities involving sample means using the CLT
5. Explain the concept of standard error and its relationship to sample size

---

## Lecture Content

### I. From Samples to Inference

In practice, we almost never observe an entire population. Instead, we draw a sample, compute a statistic such as the sample mean, and use it to estimate a population parameter. If we drew a different sample, we would get a different value of the statistic. The **sampling distribution** describes the probability distribution of a statistic over all possible samples of a given size from the population. Understanding sampling distributions is the key to understanding how inference works.

### II. Sampling Distribution of the Sample Mean

Suppose X1, X2, ..., Xn are a random sample from a population with mean mu and standard deviation sigma. The sample mean X-bar = (X1 + X2 + ... + Xn) / n has a sampling distribution with specific properties. Its mean is E(X-bar) = mu, making the sample mean an unbiased estimator of the population mean. Its standard deviation, called the **standard error**, is SE = sigma / sqrt(n). As n increases, the standard error decreases, meaning estimates become more precise. If the population itself is normal, then X-bar is exactly normally distributed for any sample size, following X-bar ~ N(mu, sigma^2 / n).

### III. The Standard Error

The standard error SE = sigma / sqrt(n) measures the variability of the sample mean from sample to sample. It is distinct from the standard deviation: SD measures the variability of individual observations, while SE measures the variability of the sample mean. A key relationship is that SE decreases with the square root of n. To halve the standard error, the sample size must be quadrupled, which creates diminishing returns -- going from n=100 to n=400 halves the SE, but to halve it again requires going from n=400 to n=1600.

In practice, sigma is usually unknown and is estimated by the sample standard deviation s, giving SE = s / sqrt(n).

<image>A figure illustrating how sample size affects the sampling distribution. Four normal curves are overlaid, all centered at the same mean (mu = 100). The curves correspond to n = 1 (widest, labeled "individual observations"), n = 10, n = 30, and n = 100 (narrowest). Each is labeled with its standard error (sigma/sqrt(n)). An annotation shows that as n increases, the distribution of X-bar becomes tighter around mu.</image>

### IV. The Central Limit Theorem (CLT)

The Central Limit Theorem is one of the most important results in all of statistics. It states that regardless of the shape of the population distribution, the sampling distribution of X-bar approaches a normal distribution as the sample size n increases. Formally, X-bar ~ approximately N(mu, sigma^2 / n) for large n. This holds even if the population distribution is skewed, uniform, bimodal, or any other shape.

How large must n be? If the population is approximately normal, any n is sufficient. If the population is moderately skewed, n >= 30 is usually adequate. If the population is highly skewed or has heavy tails, n >= 50 or more may be needed. The practical importance of the CLT cannot be overstated: it justifies the use of normal-based inference methods (z-tests, t-tests, confidence intervals) even when the underlying data are not themselves normally distributed.

### V. Demonstration of the CLT

Consider a highly right-skewed population such as the exponential distribution with lambda = 1, which has a population mean of 1 and a population standard deviation of 1. If we draw samples of various sizes and compute X-bar for each, the transformation is striking. With n = 1, the sampling distribution looks exactly like the skewed population. With n = 5, it is less skewed. With n = 30, it is approximately bell-shaped. With n = 100, it is very close to normal. This convergence to normality occurs regardless of the original distribution shape.

<image>A four-panel demonstration of the Central Limit Theorem. Top-left: The population distribution (exponential, highly right-skewed). Top-right: Histogram of 1000 sample means with n = 5 (still skewed but less so). Bottom-left: Histogram of 1000 sample means with n = 30 (approximately normal). Bottom-right: Histogram of 1000 sample means with n = 100 (very close to normal). Each histogram has a normal curve overlay with the theoretical mean and SE. The progression from skewed to normal is clearly visible.</image>

### VI. Applying the CLT: Probability Calculations

Suppose a hospital reports that the mean length of stay for a certain procedure is 4.5 days with SD = 2.0 days, and the distribution is right-skewed. For a random sample of 36 patients, what is P(X-bar > 5.0)? The standard error is SE = 2.0 / sqrt(36) = 0.333, and z = (5.0 - 4.5) / 0.333 = 1.50. Therefore P(X-bar > 5.0) = P(Z > 1.50) = 1 - 0.9332 = 0.0668. Even though individual length of stay is not normal, the CLT allows us to use the normal distribution for X-bar.

As another example, to determine the sample size needed so that P(X-bar is within 0.5 days of mu) >= 0.95, we need 0.5 / SE >= 1.96, which gives SE <= 0.255. From sigma/sqrt(n) <= 0.255, we get n >= (2.0/0.255)^2 = 61.5, so n >= 62.

### VII. Sampling Distribution of Other Statistics

The CLT applies most directly to the sample mean, but similar results hold for other statistics. The **sample proportion** (p-hat) is approximately normally distributed for large n, with E(p-hat) = p and SE(p-hat) = sqrt[p(1-p)/n], provided np >= 5 and n(1-p) >= 5. The **difference of means** (X-bar1 - X-bar2) is approximately normal for large samples, as is the **difference of proportions**. These sampling distributions form the basis for the confidence intervals and hypothesis tests covered in upcoming lectures.

### VIII. Finite Population Correction

The formulas above assume sampling with replacement or from an infinite population. When sampling without replacement from a finite population of size N, the **finite population correction (FPC)** should be applied: SE = (sigma / sqrt(n)) * sqrt((N - n) / (N - 1)). The correction factor is negligible when n/N < 0.05, meaning the sample is less than 5% of the population. In most biomedical research, populations are large enough that the FPC is not needed.

<image>A diagram summarizing the key concepts of this lecture. At the top, a non-normal population distribution (irregularly shaped). An arrow labeled "Take many samples of size n" leads to a collection of sample means. These sample means are displayed in a histogram at the bottom that is approximately normal, centered at mu with SE = sigma/sqrt(n). Annotations highlight three key results: (1) E(X-bar) = mu, (2) SE(X-bar) = sigma/sqrt(n), (3) shape approaches normal as n increases.</image>

---