Skip to main content
Logo image

Advanced Engineering Mathematics Lecture Notes: West Virginia Wesleyan College

Jesse Oldroyd

Section 2.1 Finding Eigenvalues and Eigenvectors

Many applications call for computing matrix-vector products like \(A\vb{x}\text{,}\) and in such cases it often happens that \(A\) is a square matrix. If many such products need to be computed, it'd be nice to know if there was a way to simplify these calculations. One possible way to approach this is by using eigenvectors and eigenvalues.

Definition 2.1.1. Eigenvalues and Eigenvectors.

An eigenvector of an \(n\times n\) matrix \(A\) is a nonzero vector \(\vb{x}\) such that \(A\vb{x} = \lambda\vb{x}\) for some scalar \(\lambda\text{.}\) A scalar \(\lambda\) is called an eigenvalue of \(A\) if there is a nonzero solution of the equation \(A\vb{x} = \lambda\vb{x}\text{.}\)
Note that the zero vector is not allowed to be an eigenvector, but zero is allowed to be an eigenvalue.

Example 2.1.2. Verifying Eigenvectors.

Let
\begin{equation*} A = \mqty[5 & 2 \\ 3 & 6],\qq{}\vb{v}_{1} = \mqty[1 \\ -1],\qq{}\vb{v}_{2} = \mqty[2 \\ 1]\qq{and}\vb{v}_{3} = \vb{0}. \end{equation*}
Which of the vectors, if any, is an eigenvector of \(A\text{?}\) What is one eigenvalue of \(A\text{?}\)
Solution.
First, note that \(\vb{v}_{3} = \vb{0}\) is not an eigenvector since it is the zero vector. So we'll check if the other two vectors are eigenvectors:
\begin{equation*} A\vb{v}_{1} = \mqty[3 \\ -3] = 3\mqty[1 \\ -1] \end{equation*}
and
\begin{equation*} A\vb{v}_{2} = \mqty[11 \\ 12]. \end{equation*}
So \(\vb{v}_{1}\) is an eigenvector of \(A\) (with eigenvalue \(3\)), but \(\vb{v}_{2}\) is not an eigenvector since there is no scalar we can multiply \(\vb{v}_{2}\) by to get \(A\vb{v}_{2}\text{.}\)
It's a little harder to verify if a given number is an eigenvalue of a matrix \(A\text{.}\)

Example 2.1.3. Verifying Eigenvalues.

Is \(-2\) an eigenvalue of the matrix
\begin{equation*} Q = \mqty[0 & -1 & -1 \\ -1 & 0 & -1 \\ -1 & -1 & 0]? \end{equation*}
If it is, find a corresponding eigenvector.
Solution.
\(-2\) is an eigenvalue of \(Q\) if and only if the equation \(Q\vb{x} = -2\vb{x}\) has a nonzero solution. Rearranging this equation, we can say that \(-2\) is an eigenvalue of \(Q\) if and only if \((Q+2I)\vb{x} = \vb{0}\) has a nontrivial solution. So we'll row reduce the augmented matrix \(\mqty[Q+2I & \vb{0}]\) to see if the system has free variables:
\begin{align*} \mqty[2 & -1 & -1 & 0 \\ -1 & 2 & -1 & 0 \\ -1 & -1 & 2 & 0] &\sim\mqty[-1 & 2 & -1 & 0 \\ 2 & -1 & -1 & 0 \\ -1 & -1 & 2 & 0]\\ &\rowop[2R_{1}+R_{2}]{-R_{1}+R_{3}}\mqty[-1 & 2 & -1 & 0 \\ 0 & 3 & -3 & 0 \\ 0 & -3 & 3 & 0]\\ &\sim\mqty[-1 & 2 & -1 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0] \end{align*}
so there are free variables and it follows that \(-2\) is indeed an eigenvalue of \(Q\text{.}\)
To find an eigenvector, we just need to find a nontrivial solution of \((Q+2I)\vb{x} = \vb{0}\)---equivalently, a nonzero vector in \(\nul(Q+2I)\)---so we'll continue row reducing:
\begin{align*} \mqty[-1 & 2 & -1 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0] &\rowop{-2R_{2}+R_{1}}\mqty[-1 & 0 & 1 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0]\\ &\sim\mqty[1 & 0 & -1 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0] \end{align*}
So
\begin{equation*} \nul(Q+2I) = \spn{\mqty[1 \\ 1 \\ 1]} \end{equation*}
and a single eigenvector of \(Q\) is given by
\begin{equation*} \mqty[1 \\ 1 \\ 1]. \end{equation*}

Subsection Computing Eigenvalues and Eigenvectors

There are two things we can note from Example 2.1.3. First, any nonzero vector in \(\nul(Q+2I)\) is an eigenvector of \(Q\) with eigenvalue \(-2\text{.}\) This leads to the following definition.

Definition 2.1.4. Eigenspaces.

Let \(A\) be an \(n\times n\) matrix and suppose that \(\lambda\) is an eigenvalue of \(A\text{.}\) The eigenspace of \(A\) corresponding to \(\lambda\) is the subspace of \(\RR^{n}\) containing all of the eigenvectors corresponding to \(\lambda\) in addition to the zero vector. In other words, the eigenspace of \(A\) corresponding to \(\lambda\) is the set \(\nul(A-\lambda I)\text{.}\) This set is often denoted \(E_{\lambda}\text{.}\)
The second item we note is that \(-2\) was an eigenvalue precisely because the equation \((Q+2I)\vb{x} = \vb{0}\) had nontrivial solutions. In other words, \(Q+2I\) was not invertible. In general, the polynomial \(\det(A-\lambda I)\) and equation \(\det(A-\lambda I) = 0\) are important enough that they deserve their own names.

Definition 2.1.5. Characteristic Polynomial and Characteristic Equation.

Let \(A\) be an \(n\times n\) matrix. The characteristic polynomial of \(A\) is the \(n^\th\) degree polynomial \(\det(A-\lambda I)\text{.}\) The equation \(\det(A-\lambda I) = 0\) is the characteristic equation.

Example 2.1.7. Finding Eigenvalues.

Find the eigenvalues of the matrix
\begin{equation*} A = \mqty[4 & -2 & 3 \\ 0 & -1 & 3 \\ -1 & 2 & -2]. \end{equation*}
Solution.
We need to compute the characteristic polynomial \(\det(A-\lambda I)\text{.}\) Now,
\begin{equation*} A - \lambda I = \mqty[4 - \lambda & -2 & 3 \\ 0 & -1-\lambda & 3 \\ -1 & 2 & -2 - \lambda], \end{equation*}
so
\begin{align*} \det(A-\lambda I) &= (4-\lambda)\mqty|-1-\lambda & 3 \\ 2 & -2-\lambda| - \mqty|-2 & 3 \\ -1-\lambda & 3|\\ &= (4-\lambda)\qty[(-1-\lambda)(-2-\lambda)-6] - \qty[-6-(-3-3\lambda)]\\ &= (4-\lambda)(-4+3\lambda+\lambda^{2}) - (-3+3\lambda)\\ &= (4-\lambda)(\lambda+4)(\lambda-1)+3(1-\lambda)\\ &= (\lambda-1)\qty[(4-\lambda)(\lambda+4)-3]\\ &= (\lambda-1)(13-\lambda^{2}) \end{align*}
The solutions of the characteristic equation
\begin{equation*} (\lambda-1)(13-\lambda^{2}) = 0 \end{equation*}
are given by \(\lambda = 1,\pm\sqrt{13}\text{.}\) So the eigenvalues of \(A\) are \(1,-\sqrt{13}\) and \(\sqrt{3}\text{.}\)
Computer systems such as Sage and Octave can, naturally, find eigenvalues and eigenvectors as well. In Octave this is done with the eig command. If no output is specified then the command produces an array of eigenvalues, while if two outputs are specified the command produces two matrices: the first matrix is a matrix of eigenvector columns of \(A\) and the second matrix is a diagonal matrix of eigenvalues of \(A\text{.}\) See the code cell below.
Matters are simplified greatly when finding eigenvalues of triangular matrices (see Theorem 1.6.7).

Example 2.1.8. Eigenvalues of a Triangular Matrix.

Find the eigenvalues of the matrix
\begin{equation*} B = \mqty[1 & 0 & 0 & 0 \\ 5 & 0 & 0 & 0 \\ -1 & -3 & -3 & 0 \\ 0 & 0 & 1 & -10]. \end{equation*}
Solution.
To find the eigenvalues, we need to first find \(\det(B-\lambda I)\text{.}\) Since
\begin{equation*} B-\lambda I = \mqty[1-\lambda & 0 & 0 & 0 \\ -1 & \lambda & 0 & 0 \\ -1 & -3 & -3-\lambda & 0 \\ 0 & 0 & -1 & -10-\lambda] \end{equation*}
is triangular (just as \(B\) is triangular), it follows that
\begin{equation*} \det(B - \lambda I) = (1-\lambda)\lambda(-3-\lambda)(-10-\lambda). \end{equation*}
The solutions of the characteristic equation, and thus the eigenvalues of \(B\text{,}\) are given by \(\lambda = 1, 0, -3, -10\text{.}\)
Example 2.1.8 suggests the following theorem.
So now we have a good idea of how to find eigenvalues of a square matrix \(A\text{:}\) just solve the characteristic equation \(\det(A-\lambda I) = 0\text{.}\) To find the corresponding eigenvectors, we need to solve the related equation \(A\vb{v} = \vb{0}\text{,}\) which reduces to solving \((A-\lambda I)\vb{v} = \vb{0}\text{.}\)

Example 2.1.10. Finding Eigenvalues and Eigenvectors.

Find the eigenvalues and eigenvectors of
\begin{equation*} A = \mqty[2 & 0 & -1 \\ 0 & \frac{1}{2} & 0 \\ 1 & 0 & 4]. \end{equation*}
Solution.
First, we need to find the eigenvalues. Since
\begin{equation*} \det(A-\lambda I) = \mqty|2-\lambda & 0 & -1 \\ 0 & \frac{1}{2}-\lambda & 0 \\ 1 & 0 & 4-\lambda| = (2-\lambda)\qty(\frac{1}{2}-\lambda)(4-\lambda) + \frac{1}{2}-\lambda \end{equation*}
which simplifies to
\begin{equation*} \det(A-\lambda I) = \qty(\frac{1}{2}-\lambda)\left[(2-\lambda)(4-\lambda)+1\right] = \qty(\frac{1}{2}-\lambda)[\lambda^{2}-6\lambda+9], \end{equation*}
we see that the eigenvalues of \(A\) are given by \(\lambda=\frac{1}{2},3\text{.}\)
Now we can start trying to find eigenvectors. First, we'll row reduce \(\mqty[A-\frac{1}{2}I & 0]\) to find an eigenvector corresponding to \(\lambda=\frac{1}{2}\text{:}\)
\begin{align*} \mqty[\frac{3}{2} & 0 & -1 & 0 \\ 0 & 0 & 0 & 0 \\ 1 & 0 & \frac{7}{2} & 0] &\rowop[2R_{3}]{2R_{1}}\mqty[3 & 0 & -2 & 0 \\ 0 & 0 & 0 & 0 \\ 2 & 0 & 7 & 0]\\ &\rowop{2R_{3}+R_{1}}\mqty[1 & 0 & -9 & 0 \\ 0 & 0 & 0 & 0 \\ 2 & 0 & 7 & 0]\\ &\rowop{-2R_{1}+R_{3}}\mqty[1 & 0 & -9 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 25 & 0]\\ &\sim\mqty[1 & 0 & -9 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0]\\ &\rowop{9R_{3}+R_{1}}\mqty[1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0]. \end{align*}
Here's what this is saying: if \(\vb{v} = \mqty[\xmat*{v}{1}{3}]^{T}\) is an eigenvector of \(A\) corresponding to \(\lambda=\frac{1}{2}\text{,}\) or in other words a nontrivial solution of \((A-\frac{1}{2}I)\vb{v} = \vb{0}\text{,}\) then we must have \(v_{1} = v_{3} = 0\) and \(v_{2}\) free. So one nonzero eigenvector corresponding to \(\lambda=\frac{1}{2}\) is given by
\begin{equation*} \vb{v}_{1} = \mqty[0\\1\\0]. \end{equation*}
Therefore \(\smqty[0\\1\\0]\) forms a basis of the eigenspace \(E_{\frac{1}{2}}\) of \(A\text{.}\)
To find an eigenvector corresponding to \(\lambda = 3\text{,}\) we'll now row reduce \(\mqty[A - 3I & \vb{0}]\text{:}\)
\begin{equation*} \mqty[-1 & 0 & -1 & 0 \\ 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0]\sim\mqty[0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0]. \end{equation*}
So if \(\vb{v}\) is an eigenvector corresponding to the eigenvalue \(\lambda=3\text{,}\) then we need \(v_{1} = -v_{3}\text{,}\) and \(v_{2} = 0\text{.}\) Which means that
\begin{equation*} \vb{v} = \mqty[\xmat*{v}{3}{1}] = \mqty[-v_{3} \\ 0 \\ v_{3}] = v_{3}\mqty[-1\\0\\1]. \end{equation*}
The vector \(\smqty[-1\\0\\1]\) therefore forms a basis of the eigenspace \(E_3\) of \(A\text{.}\)
As mentioned above, the computations in Example 2.1.10 can also be carried out using technology. In Octave, the computation proceeds using eig:
Octave by design will give eigenvectors that have unit norm (i.e., magnitude of \(1\)). In the above cell the first two columns of \(U\) are eigenvectors in \(E_3\text{,}\) while the last column is an eigenvector in \(E_{\frac{1}{2}}\text{.}\) In Subsection , we examine the reason why \(3\) shows up twice in the matrix \(D\) of eigenvalues (and why a column of \(U\) is repeated here).
As Octave is designed for numerical work, it's a little awkward to try to get it to produce symbolic answers. For symbolic mathematics, a system such as Sage (or another CAS) is more appropriate. In Sage, eigenvalues can be found like so:
Octave-like output can also be produced using the eigenmatrix_right() method:

Subsection Algebraic and Geometric Multiplicities of Eigenvalues

There are a couple of interesting things we can note about the Example 2.1.10. First, each eigenvalue had at least one corresponding eigenvector. Second, \(\lambda=3\) was basically a “repeated” eigenvalue, since it showed up as a double root in the characteristic equation \(\det(A-\lambda I)=0\text{.}\) This leads us to some terminology.

Definition 2.1.11. Algebraic and Geometric Multiplicity.

Let \(A\) be an \(n\times n\) matrix and let \(\lambda\) be an eigenvalue of \(A\text{.}\) The algebraic multiplicity of \(\lambda\) is defined to be the multiplicity of \(\lambda\) as a root of the characteristic equation. The geometric multiplicity of \(\lambda\) is defined to be the number of linearly independent eigenvectors corresponding to \(\lambda\text{.}\) Equivalently, the geometric multiplicity is exactly the dimension of the eigenspace corresponding to \(\lambda\text{:}\) \(\dim E_{\lambda}\text{.}\)
In Example 2.1.10, \(\lambda = 3\) had an algebraic multiplicity of \(2\text{.}\) This could also be seen in the Octave and Sage results following the example, since \(3\) showed up twice in the diagonal matrix of eigenvalues. This was also given after using eigenvectors_right() immediately after the given eigenvector for \(\lambda = 3\text{.}\) On the other hand, the geometric multiplicity of \(\lambda = 3\) was \(\dim E_{3} = 1\text{.}\) This was represented in the repeated eigenvector in \(U\) in the Octave result and the single nonzero vector corresponding to \(\lambda = 3\) in the Sage result. In this case, we say that \(A\) is defective.

Definition 2.1.12. Defective Matrices.

Let \(A\) be a square matrix. If the sum of the geometric multiplicities of the eigenvalues of \(A\) is less than the sum of algebraic multiplicities of eigenvalues of \(A\text{,}\) then we say that \(A\) is defective.
The following result gives some basic estimates for algebraic and geometric multiplicities of eigenvalues.
From Theorem 2.1.13, an \(n\times n\) square matrix \(A\) is defective if and only if it has at most \(n-1\) linearly independent eigenvectors. Equivalently, the eigenvectors of \(A\) are not enough to form a basis (also known as an eigenbasis) of \(\RR^n\text{.}\) This leads to difficulties in computations involving \(A\text{.}\)

Definition 2.1.14. Eigenbases.

Let \(A\) be an \(n\times n\) square matrix that is not defective. A set \(\qty{\vb{v}_1, \ldots, \vb{v}_n}\) is called an eigenbasis for \(\RR^n\) if each vector in the set is an eigenvector of \(A\) and if the set also forms a basis of \(\RR^n\text{.}\)

Example 2.1.15. Multiplicities and Eigenspaces.

Graph the eigenspaces of the matrix
\begin{equation*} A = \mqty[1 & 3 \\ 0 & 0]. \end{equation*}
What are the algebraic and geometric multiplicities of each eigenvalue?
Solution.
Since \(A\) is triangular, its eigenvalues are \(0\) and \(1\text{.}\) We also know at this point that \(A\) can't be defective. In particular, we can form a basis of \(\RR^2\) entirely from eigenvectors of \(A\text{.}\) To actually find the eigenvectors, we need to find the corresponding eigenspaces \(E_{0} = \nul(A-0I) = \nul A\) and \(E_{1} = \nul(A-I)\text{.}\)
By inspection,
\begin{equation*} \nul A = \spn{\mqty[-3 \\ 1]}. \end{equation*}
The eigenspace associated with \(1\) is \(\nul(A-I)\text{.}\) Since
\begin{equation*} A- I - \mqty[0 & 3 \\ 0 & -1] \end{equation*}
we see that
\begin{equation*} \nul(A-I) = \spn{\mqty[1\\0]}. \end{equation*}
So the graph of the eigenspaces is given by
A plot of the eigenspaces of a matrix
Figure 2.1.16. Eigenspaces of \(A\)
A corresponding eigenbasis from \(A\) would be the set \(\qty{\smqty[-3\\1], \smqty[1\\0]}\text{.}\)
The importance of an eigenbasis is demonstrated in the next example.

Example 2.1.17. Matrix Multiplication and Eigenvectors.

Let
\begin{equation*} \mathcal{B} = \qty{\vb{b}_{1},\vb{b}_{2}} = \qty{\mqty[-3 \\ 1], \mqty[1 \\ 0]} \end{equation*}
and let \(A\) be as in the previous example. Suppose that \(\vb{v}\) is a vector in \(\RR^{2}\) such that
\begin{equation*} \vb{v} = 4\vb{b}_{1}+3\vb{b}_{2}. \end{equation*}
Compute \(A\vb{v}\text{.}\)
Solution.
This computation will be quite easy. Since \(\vb{v} = 4\vb{b}_{1}+3\vb{b}_{2}\text{,}\) we have
\begin{equation*} A\vb{v} = 4A\vb{b}_{1}+3A\vb{b}_{2} = 3\vb{b}_{2} = \mqty[3 \\ 0]. \end{equation*}
<Prev^TopNext>
Authored in PreTeXt
Powered by Runestone