6.1 Eigenvalues and Eigenvectors

1Reading¶

Material related to this page, as well as additional exercises, can be found in ALA 8.1.

2Learning Objectives¶

By the end of this page, you should know:

the definition of the eigenvalues and eigenvectors of a square matrix $A$ ,
how to find the eigenvectors corresponding to an eigenvalue.

3Eigenvalues and Eigenvectors¶

In this chapter, we will discuss one of the most fundamental elements of linear algebra: the eigenvalues and eigenvectors of a square matrix $A$ . Eigenvalues and eigenvectors are ubiquitous throughout linear algebra and calculus, and in later sections we’ll discuss their applications to the analysis of linear dynamical systems (sections 6.8 and 7.1-7.5).

Geometrically, when $A$ acts on an eigenvector $\vv v$ , it does not change its orientation: it only stretches it by the value specified by the eigenvalue λ. In a general sense, the eigenvectors and eigenvalues can be used to describe $A$ by indicating how it is “stretching” a vector space.

Our study of eigenvalues and -vectors will begin with a discussion on how we find them.

3.1Finding the eigenvectors for a known eigenvalue¶

We’ll first consider an easier problem.

Suppose we already know that $A$ has an eigenvalue λ. Then, (EIG) is a linear system in $\vv v$ . If we isolate our vector $\vv v$ by re-arranging terms, we get

(A - \lambda I)\mathbf{v} = \mathbf{0}

(2)

This is a system we know how to solve. Specifically, the solution set is the null space of $A - \lambda I$ .

Exercise 1 (Finding eigenvectors)

Given that $\lambda = -3$ is an eigenvalue of $\bm -1 & -1 & 1 \\ -4 & -1 & -2 \\ 0 & 0 & -3 \em$ , find its corresponding eigenvectors.

Solution to Exercise 1

To find the corresponding eigenvectors, we solve the homogenous system:

\begin{align*} \bm -1 & -1 & 1 \\ -4 & -1 & -2 \\ 0 & 0 & -3 \em - (-3)I = \bm 2&-1&1\\-4&2&-2\\0&0&0 \em = \vv 0. \end{align*}

(3)

After forming the augmented matrix and putting it in row echelon form, we get the equivalent system:

\begin{align*} \left[\begin{array}{ccc|c} 2&-1&1 &0\\ 0&0&0 &0\\ 0&0&0 &0 \end{array}\right] \end{align*}

(4)

which has solution set given by (and hence the eigenvectors corresponding to an eigenvalue of -3 are given by):

\begin{align*} \text{span}\left\{ \bm 1/2\\1\\0\em, \bm -1/2\\0\\1 \em \right\}. \end{align*}

(5)

Python Break!¶

Below is a code snippet demonstrating how to solve for the eigenvectors of a known eigenvalue. We aren’t introducing anything new, we are just applying the concepts we have covered in the previous few chapters!

import numpy as np 
from scipy import linalg

A = np.array([
    [-1, -1, 1],
    [-4, -1, -2],
    [0, 0, -3]
])

lambda_1 = -3        # we are given that -3 is an eigenvalue of A

print('Eigenvectors corresponding to lambda_1 = -3:')
print(linalg.null_space(A - (-3) * np.identity(3)))

Here, the null_space function from the scipy.linalg library returns a matrix whose columns form an orthonormal basis of a given matrix $A$ . To solve for the eigenvectors corresponding to $\lambda_1 = -3$ , use the null_space function to solve for the nullspace of $A - (-3)I$ .

4... But how do we find the eigenvalues?¶

The question, now, becomes how to find the eigenvalues of a matrix. A key observation is that the definition of eigenvalue requires that the corresponding eigenvector be nonzero, and we know that this can only occur if $A - \lambda I$ is singular! (Recall that non-singular matrices only have the $\vv 0$ vector in the null-space). This discussion is summarized in the following theorem:

This theorem gives us a plan of attack: to find all λ for which $A - \lambda I$ is singular!

In the next few sections, we’ll introduce the machinery needed to apply this method, known as the method of characteristic equations, for finding eigenvalues of small matrices.

In general, this method is not applied for large matrices due to numerical issues. For larger matrices, an algorithm based on the QR decomposition is used; we’ll cover this a few sections down the line.