Premed · Premed · Calculus 1
Lecture 17: Newton's Method
Calculus I — Differential Calculus
Learning Objectives
By the end of this lecture, students will be able to:
- Describe the geometric idea behind Newton's Method
- Derive and apply the Newton's Method iteration formula
- Use Newton's Method to approximate roots of equations
- Identify situations where Newton's Method fails or converges slowly
- Understand the rate of convergence of Newton's Method
Lecture Content
I. Motivation
Many equations cannot be solved algebraically. Equations like x^5 + 3x - 1 = 0 or cos x = x have no closed-form solution. The Intermediate Value Theorem can confirm that a root exists in some interval, but it does not provide the value. Newton's Method (also called the Newton-Raphson Method) is an iterative numerical technique for approximating roots with high accuracy. It leverages the derivative to generate successively better approximations.
II. The Geometric Idea
The goal is to find a root of f(x) = 0. Start with an initial guess x_0 near the root. Draw the tangent line to y = f(x) at the point (x_0, f(x_0)). This tangent line intersects the x-axis at a new point x_1, which should be closer to the root than x_0 was. Now repeat: draw the tangent at (x_1, f(x_1)), find where it hits the x-axis to get x_2, and continue iterating. Each step typically brings the approximation closer to the true root.
<image>A graph showing Newton's Method geometrically. A curve y = f(x) crosses the x-axis at a root r. Starting from an initial guess x_0 on the x-axis, a vertical line goes up to the curve at (x_0, f(x_0)). The tangent line at that point is drawn, intersecting the x-axis at x_1. From x_1, another vertical line goes to the curve, and a new tangent line is drawn, intersecting at x_2. Each successive x-value is closer to the root r. The zigzag pattern of tangent lines is clearly shown. Title: "Newton's Method: tangent lines converge to the root."</image>
III. The Iteration Formula
The tangent line at (x_n, f(x_n)) has equation y - f(x_n) = f'(x_n)(x - x_n). Setting y = 0 to find where it crosses the x-axis gives 0 - f(x_n) = f'(x_n)(x_{n+1} - x_n), which rearranges to the Newton's Method formula: x_{n+1} = x_n - f(x_n)/f'(x_n), for n = 0, 1, 2, .... This formula requires that f'(x_n) is nonzero at each step.
IV. Worked Example — Finding sqrt(2)
To find sqrt(2), solve x^2 - 2 = 0. With f(x) = x^2 - 2 and f'(x) = 2x, the iteration formula becomes x_{n+1} = x_n - (x_n^2 - 2)/(2x_n) = (x_n + 2/x_n)/2. Starting with x_0 = 1, the iterates are x_1 = 1.5, x_2 = 1.41667, x_3 = 1.41422..., and x_4 = 1.41421356..., which is correct to 8 decimal places. The convergence is extremely rapid, with the number of correct digits roughly doubling at each iteration.
V. Worked Example — cos x = x
To solve cos x = x, define f(x) = cos x - x, so f'(x) = -sin x - 1. The iteration formula is x_{n+1} = x_n - (cos x_n - x_n)/(-sin x_n - 1). Starting with x_0 = 0.5 gives x_1 approximately equals 0.7553, x_2 approximately equals 0.7391, and x_3 approximately equals 0.7391 (converged). The fixed point is approximately x = 0.7391.
<image>Panel A: The graphs of y = cos x and y = x on the same axes, intersecting at approximately x = 0.739. The intersection point is highlighted. Panel B: A table showing Newton's Method iterations: n, x_n, f(x_n), f'(x_n) for n = 0 through 4, with the values converging to 0.739085... The rapid convergence is visible in the shrinking f(x_n) values. Title: "Newton's Method applied to cos x = x."</image>
VI. When Newton's Method Fails
Newton's Method can fail in several ways. If f'(x_n) = 0 at any step, the tangent line is horizontal and never intersects the x-axis, causing the method to break down. A poor initial guess may cause the iterates to diverge or oscillate rather than converge; for example, f(x) = x^{1/3} at x_0 other than 0 produces iterates that diverge. Cycling can occur when the iterates bounce back and forth without converging. At multiple roots (where f'(r) = 0 at the root r), convergence is slower -- linear rather than quadratic. Discontinuities or sharp bends may cause the method to jump to an unintended root. Best practice is to use the IVT or a graph to choose x_0 close to the actual root.
VII. Convergence Properties
When Newton's Method converges, it typically does so quadratically: the error at step n+1 is proportional to the square of the error at step n. Formally, |x_{n+1} - r| is approximately C * |x_n - r|^2 for some constant C. This means the number of correct decimal digits approximately doubles with each iteration.
For comparison, the bisection method is guaranteed to converge but is slow, exhibiting linear convergence and gaining about one decimal place of accuracy per three iterations. Newton's Method is far faster when it works, but unlike bisection, it is not guaranteed to converge. In practice, Newton's Method is the workhorse of numerical root-finding, and understanding both its power and its limitations is essential.
<image>A convergence comparison chart. Panel A: A table comparing bisection method and Newton's method for finding sqrt(2), showing the approximation at each iteration. Bisection takes about 10 iterations for 3 decimal places; Newton takes 3-4 iterations for 8 decimal places. Panel B: A log-scale plot of |error| vs. iteration number for both methods, showing the linear convergence of bisection (straight line) vs. the quadratic convergence of Newton (rapidly plunging curve). Title: "Convergence rates: bisection vs. Newton's method."</image>
