Cholesky, Doolittle and Crout Factorization
Background
Definition ( LU-Factorization ). The
nonsingular matrix A has an LU-factorization if it can be expressed as
the product of a lower-triangular matrix L and an upper triangular matrix
U:
.
When this is possible we say that A has an LU-decomposition. It
turns out that this factorization (when it exists) is not unique. If L
has 1's on it's diagonal, then it is called a Doolittle factorization. If U
has 1's on its diagonal, then it is called a Crout factorization. When (or
), it
is called a
Cholesky decomposition.
Theorem (A =
LU; Factorization with
NO Pivoting). Assume
that A has a Doolittle, Crout or Cholesky factorization. The
solution X to the linear system ,
is found in three steps:
1. Construct
the matrices ,
if possible.
2. Solve for using
forward substitution.
3. Solve for using
back substitution.
Theorem (A = LU; Doolittle
Factorization). Assume that A
has a Doolittle factorization A = LU.
=
The solution X to the linear system ,
is found in three steps:
1. Construct
the matrices ,
if possible.
2. Solve for using
forward substitution.
3. Solve for using
back substitution.
For curiosity, the reader might be interested in other methods of computing L and U.
Theorem (A = LU; Crout
Factorization). Assume that A
has a Crout factorization A = LU.
=
The solution X to the linear system ,
is found in three steps:
1. Construct
the matrices ,
if possible.
2. Solve for using
forward substitution.
3. Solve for using
back substitution.
Mathematica Subroutine (Doolittle).
Mathematica Subroutine (Crout).
Mathematica Subroutine (Forward Elimination).
Mathematica Subroutine (Back Substitution).
Theorem ( Cholesky
Factorization ). If A
is real, symmetric and positive definite matrix, then it has a Cholesky
factorization
where U an upper triangular matrix.
Remark. Observe that
is a lower triangular matrix, so that A = LU. Hence we could
also write Cholesky factorization
where L a lower triangular matrix.
Theorem (A = LU; Cholesky
Factorization). Assume that A
has a Cholesky factorization , where .
=
Or if you prefer to write the Cholesky
factorization as , where
.
=
The solution X to the linear system ,
is found in three steps:
1. Construct
the matrices ,
if possible.
2. Solve for using
forward substitution.
3. Solve for using
back substitution.
The following Cholesky subroutine can be used when the matrix A is
real, symmetric and positive definite.
Observe that the loop starting with For[j=k,j<=n,j++, is
not necessary and that U is computed by forming the transpose of L.
Mathematica Subroutine (Cholesky factorization).
Application to Polynomial Curve Fitting
Theorem ( Least-Squares
Polynomial Curve Fitting ).
Given the data
points , the
least squares polynomial of degree m of the
form
that fits the n data points is obtained by
solving the following linear system
for the m+1 coefficients
. These
equations are referred to as the "normal equations".
Application to Continuous Least Squares
Approximation
The continuous least squares approximation to a function
on the interval [0,1] for the set of functions can
solved by using the normal equations
(1) for .
Where the inner product is
. Solve
the linear system (1) for the coefficients
and construct the approximation function
.
Definition ( Gram
Matrix ). The Gram matrix
G is a matrix of inner products where the elements are .
The case when the set of functions is will
produce the Hilbert matrix. Since we require the computation to be as exact as
possible and an exact formula is known for the inverse of the Hilbert matrix,
this is an example where an inverse matrix comes in handy.
|