From the first expression,
From the second expression
Usual practise to determine an eigenvector with length one. In the example:
Definition: Quadratic form Q(x) in k variables
is defined as
Q(x) = x'Ax where A is a fix (k × k) matrix
The quadratic form is a quadratic function of .
Theorem Let A be a (k × k) symmetric matrix, (i. e. A =
A') then A has k pairs of
eigenvalues and eigenvectors
The eigenvectors can be chosen to satisfy
and be mutually perpendicular. The eigenvectors are unique
unless two or more eigenvalues
Definition: Positive Definite Matrices Let A be a symmetric matrix (k × k). A is
said to be positive definite if
x'Ax > 0 for all x ∈ Rk x ≠ 0.
A is positive semi-definite if
x'Ax ≥ 0 for all x ∈ Rk.
Theorem: Spectral decomposition of a symmetric (k × k) matrix A is given:
are eigenvalues of A
the associated normalized eigenvectors.
Theorem A, (k × k) symmetric matrix is positive definite if and only if every eigenvalue
of A is positive.
A is positive semi-definite if and only if every eigenvalue of A is nonnegative.
Proof: Trivial from the spectral decomposition theorem.
where . Choosing x = , j = 1, . . . , k) follows the theorem.
Another form of the spectral decomposition:
INVERZE AND SQUARE ROOT OF A POSITIVE DEFINIT MATRIX
By the spectral decomposition, if A is positive definite:
Inverse of A If A is positive definite, then the eigenvaluse of A are
i = 1, . . . , k. The inverse of A is where .
Square Root Matrix (defined for positive semi-definite A)
MATRIX INEQUALITIES AND MAXIMIZATION
Cauchy-Schwarz Inequality: Let b, d ∈ Rp, then
with equality if and only if b = cd.
Proof: The inequality is obvious when one of the vectors is the zero one. For b − xd ≠ 0 we
Adding and substratcting , we have that
Choosing x = b'd/d'd follows the statement.
Extended Cauchy–Schwartz Inequality: Let b, d ∈ Rp and let B be a positive definite
(p × p) matrix. Then
with equality if and only if b = cB-1d for some constant c.
Maximization of Quadratic forms on the Unit Sphere:
Let B be a (p × p) positive definite matrix with eigenvectors and associated
eigenvalues eigenvectors, where . Then
and the equality is attained when .