7.4 Invariant Subspaces - ESE 2030 📏

1Reading¶

Material related to this page, as well as additional exercises, can be found in ALA 8.4 and 10.4.

2Learning Objectives¶

By the end of this page, you should know:

the definition and some examples of invariant subspaces
the invariant subspaces spanned by the eigenvectors of the matrix
the different invariant subspaces of a complete matrix and how they relate to the behavior of linear dynamical systems

3Definition and Examples¶

Invariant subspaces of linear maps play a key role in dynamical systems, linear iterative systems (like Markov chains, which we’ll see next lecture), and control systems. Perhaps not surprisingly, the theory of invariant subspaces is built on eigenvalues and eigenvectors.

We start by defining an invariant subspace with respect to a linear transformation.

Intuitively, an invariant subspace $W\subset V$ is like Vegas: what happens in $W$ stays in $W$ ! Let’s see some simple examples before developing a more general theory.

4Subspaces Spanned by the Eigenvectors¶

Since we will focus on cases where $V=\mathbb{R}^n$ , our linear transformations will be defined by matrices $A:\mathbb{R}^n \to \mathbb{R}^n$ : $L(\vv x)=A\vv x$ . In this case, we can characterize 1-d invariant subspaces very clearly.

We won’t formally prove this result, but instead give a hint as to why it might be true. Suppose $W = \text{span}\{\vv v_1,\ldots,\vv v_k\}$ for $\vv v_1,\ldots,\vv v_k$ linearly independent eigenvectors of $A$ . Then any $\vv w \in W$ can be written as

\vv w = c_1\vv v_1 + \cdots + c_k\vv v_k

(2)

for $c_1,\ldots,c_k \in \mathbb{C}$ . Then

A \vv w = A(c_1\vv v_1 + \cdots + c_k\vv v_k) = c_1A\vv v_1 + \cdots + c_kA\vv v_k = c_1\lambda_1\vv v_1 + \cdots + c_k\lambda_k\vv v_k \in \text{span}\{\vv v_1,\ldots,\vv v_k\} = W

(3)

and hence $W$ is invariant under the map $\vv x \mapsto A\vv x$ . The challenge is to show the other direction, that if $W$ is invariant under $A$ then $W$ must be spanned by $k$ eigenvectors of $A$ . We refer interested readers to proof of Thm 8.3 in ALA 8.4.

Note

Some final comments before we see an example:

If $A$ is a complex real matrix with all real eigenvalues, then the above tells us that all real invariant subspaces are spanned by eigenvectors of $A$ .
If $A$ is real and complete, and has complex conjugate eigenvectors $\vv v_t = \vv x \pm i \vv y$ , then the real invariant subspaces are spanned by $\text{Re}\{\vv v_t\} = \vv x$ and $\text{Im}\{\vv v_t\} = \vv y$ using a similar argument as to the one we used to find real solutions to $\dot{\vv x} = A\vv x$ when $A$ had complex conjugate eigenvalues.
A slightly modified argument can be applied to incomplete matrices using Jordan blocks and generalized eigenvectors; we can’t cover these in this course, but if you’re curious, you can check out ALA 8.6.

Example 5

Consider the rotation (permutation) matrix

A = \bm 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \em

(4)

It has one real eigenvalue, $\lambda_1 = 1$ , and two complex conjugate eigenvalues, $\lambda_2 = \frac{1}{2} + \frac{\sqrt{3}}{2}i$ and $\lambda_3 = \frac{1}{2} - \frac{\sqrt{3}}{2}i$ . The corresponding eigenvectors are:

\vv v_1 = \bm 1 \\ 1 \\ 1 \em, \quad \vv v_2 = \bm -\frac{1}{2} \\ -\frac{1}{2} \\ 1 \end{bmatrix} + i\bm \frac{\sqrt{3}}{2} \\ -\frac{\sqrt{3}}{2} \\ 0\em, \quad \vv v_3 = \bm -\frac{1}{2} \\ -\frac{1}{2} \\ 1 \em - i\bm \frac{\sqrt{3}}{2} \\ -\frac{\sqrt{3}}{2} \\ 0\em

(5)

The complex invariant subspaces are spanned by $0, 1, 2$ or 3 of $\vv v_1$ , $\vv v_2$ , or $\vv v_3$ . There is a single 1d real invariant subspace spanned by $\vv v_1 = \bm 1 \\ 1 \\ 1\em$ , and a single 2d real invariant subspace spanned by $\text{Re}\{\vv v_2\} = \bm -\frac{1}{2} \\ -\frac{1}{2} \\ 1\em$ and $\text{Im}\{\vv v_2\} = \bm \frac{\sqrt{3}}{2} \\ -\frac{\sqrt{3}}{2} \\ 0 \em$ which is the orthogonal complement to $\vv v_1$ . We can interpret $\vv v_1$ as the axis of rotation, and $A$ acts as a 2d rotation on its orthogonal complement.

4.1Plot for Example 5¶

An interactive 3d plot is given below along with the python code that illustrate the primary vectors and the subspace in Example 5. The orthogonal vectors are $\vv v_2$ ( $\textcolor{red}{red}$ ) and $\vv v_3$ ( $\textcolor{green}{green}$ ) that span the 2d invariant subspace (in $\textcolor{cyan}{cyan}$ ). The axis of rotation is $\vv v_1$ ( $\textcolor{blue}{blue}$ ). We used Plotly to plot the below figure, which is a nice tool to plot interactive figures on google colab/jupyter notebook.

import numpy as np
import plotly.graph_objects as go
import ipywidgets as widgets
from IPython.display import display

## Define the orthogonal vectors and the axis of rotation

# Orthogonal vectors
v1 = np.array([-1/2, -1/2, 1]) # v2 in the exercise
v2 = np.array([np.sqrt(3)/2, -np.sqrt(3)/2, 0]) # v3 in the exercise

axis = np.array([1, 1, 1]) # axis of rotation: # v1 in the exercise

# Define the plane
u = np.linspace(-1, 1, 10)
v = np.linspace(-1, 1, 10)
U, V = np.meshgrid(u, v)
X = U * v1[0] + V * v2[0]
Y = U * v1[1] + V * v2[1]
Z = U * v1[2] + V * v2[2]

# Create the 3D plot
fig = go.Figure()

# Add vectors to the plot without markers
fig.add_trace(go.Scatter3d(x=[0, v1[0]], y=[0, v1[1]], z=[0, v1[2]],
                            mode='lines', line=dict(color='red', width=8)))
fig.add_trace(go.Scatter3d(x=[0, v2[0]], y=[0, v2[1]], z=[0, v2[2]],
                            mode='lines', line=dict(color='green', width=8)))
fig.add_trace(go.Scatter3d(x=[0, axis[0]], y=[0, axis[1]], z=[0, axis[2]],
                            mode='lines', name='axis', line=dict(color='blue', width=8)))

# Add the plane
fig.add_trace(go.Surface(x=X, y=Y, z=Z, colorscale=[[0, 'cyan'], [1, 'cyan']], opacity=0.5, showscale=False))

# Add arrowheads (cones) at the end of the vectors
fig.add_trace(go.Cone(x=[v1[0]], y=[v1[1]], z=[v1[2]],
                      u=[v1[0]], v=[v1[1]], w=[v1[2]],
                      colorscale=[[0, 'red'], [1, 'red']], sizemode='absolute', sizeref=0.3, anchor="tail", showscale=False, showlegend=False))
fig.add_trace(go.Cone(x=[v2[0]], y=[v2[1]], z=[v2[2]],
                      u=[v2[0]], v=[v2[1]], w=[v2[2]],
                      colorscale=[[0, 'green'], [1, 'green']], sizemode='absolute', sizeref=0.3, anchor="tail", showscale=False, showlegend=False))
fig.add_trace(go.Cone(x=[axis[0]], y=[axis[1]], z=[axis[2]],
                      u=[axis[0]], v=[axis[1]], w=[axis[2]],
                      colorscale=[[0, 'blue'], [1, 'blue']], sizemode='absolute', sizeref=0.3, anchor="tail", showscale=False, showlegend=False))

# Set the layout
fig.update_layout(scene=dict(
                    xaxis=dict(range=[-1.5, 1.5]),
                    yaxis=dict(range=[-1.5, 1.5]),
                    zaxis=dict(range=[-1.5, 1.5]),
                    aspectmode='cube'),
                  width=700, height=700)

Loading...

5Invariant Subspaces and Linear Dynamical Systems¶

Here we give a very brief preview of the role of invariant subspaces in dynamical systems. You will see this in much more detail in ESE 2100.

We call a subset $S \subset \mathbb{R}^n$ invariant for $\dot{\vv x} = Ax$ if, whenever $\vv x(0) = \vv b \in S$ then the solution $\vv x(t) \in S$ for all $t \geq 0$ . It turns out, invariant subspaces of $A$ precisely characterize these subsets:

Proposition 2

If $S \subset \mathbb{R}^n$ is an invariant subspace of $A$ , then it is invariant under $\dot{\vv x} = Ax$ .

Proof 2 (Proof of Proposition 2)

The proof follows from our solution $\vv x(t) = e^{At}\vv x(0) = e^{At}\vv b$ . Using (MPS), we have:

\vv x(t) = e^{At}\vv b = \sum_{k=0}^{\infty} \frac{t^k}{k!} A^k \vv b.

(6)

But if $\vv b \in S$ , then $A\vv b \in S$ , $A^2\vv b \in S$ , and in general $A^k\vv b \in S$ for any $k \geq 0$ . Since every term in (6) belongs to $S$ , then so does their (infinite) sum¹, hence $x(t) \in S$ .

¹ This is because a subspace is a closed set, and so we are allowed to take infinite sums.

We will focus on the case of complete matrices $A$ with real eigenvalues and eigenvectors; extensions to the general case are similar, and rely on using Jordan blocks and taking Real/Imaginary parts of complex eigenvectors.

These subspaces are important, as they describe the long-term behavior of solutions with initial conditions within them. Before elucidating this observation, we make the following comments:

Remembering that to each eigenvalue/vector pair $(\lambda_i, \vv v_i)$ we can associate an eigenfunction $\vv x_i(t) = e^{\lambda_i t}\vv v_i$ , we can characterize the following long-term behavior of solutions to $\dot{\vv x} = A\vv x$ :

This theorem tells us from what subsets of $\mathbb{R}^n$ we should pick initial conditions $\vv x(0) = \vv b$ if we want our solutions to decay to zero (stable), not move (center), or blow up to infinity (unstable). This has very important applications in analyzing the behavior of dynamical systems, which we’ll explore in the case study.

Example 6

The matrix $A = \begin{bmatrix} -2 & 1 & 0 \\ 1 & -1 & 1 \\ 0 & 1 & -2 \end{bmatrix}$ has eigenvalue/vector pairs

\lambda_1 = 0, \vv v_1 = \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}, \quad \lambda_2 = -2, \vv v_2 = \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}, \quad \lambda_3 = -3,\vv v_3 = \begin{bmatrix} 1 \\ -1 \\ 1 \end{bmatrix}.

(7)

Thus the stable subspace is spanned by $\vv v_2$ and $\vv v_3$ , whose nonzero solutions tend to $\vv 0$ as $t \to \infty$ (exponentially quickly); the center subspace is the line spanned by $\vv v_1$ , all of whose solutions are constant. In this case, no eigenvalues are positive and hence the unstable subspace is trivial: $U = \{\vv 0\}$ .