7.3 Matrix Exponential

1Reading¶

Material related to this page, as well as additional exercises, can be found in ALA 10.4.

2Learning Objectives¶

By the end of this page, you should know:

how to define the matrix exponential as a power series,
how to solve linear ODEs with the matrix exponential,
how to compute the matrix exponential.

3Defining the Matrix Exponential¶

We’ve seen four cases for eigenvalues/eigenvectors and their relationship to solutions of initial value problems defined by $\dot{\mathbf{x}} = A\mathbf{x}$ and $\mathbf{x}(0)$ given:

real distinct eigenvalues, solved by diagonalization;
real repeated eigenvalues with algebraic multiplicity = geometric multiplicity, also solved by diagonalization;
complex distinct eigenvalues, solved by diagonalization and applying Euler’s formula to define real-valued eigenfunctions;
repeated eigenvalues with algebraic multiplicity > geometric multiplicity, solved by Jordan decomposition using generalized eigenvectors.

While correct, the fact that there are four different cases we need to consider is somewhat unsatisfying. In this section, we show that by appropriately defining a matrix exponential, we can provide a unified treatment of all the aforementioned settings.

We start by recalling the power series definition for the scalar exponential $e^x$ , for $x \in \mathbb{R}$ :

e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots = \sum_{k=0}^{\infty} \frac{x^k}{k!}, \quad (\text{PS})

(1)

where we recall that $k! = 1 \cdot 2 \cdots (k-1) \cdot k$ . We know that for the scalar initial value problem $\dot{x} = ax$ , the solution is $x(t) = e^{at}x(0)$ , where $e^{at}$ can be computed via (PS) by setting $x = at$ .

Wouldn’t it be cool if we could do something similar for the vector valued initial value problem defined by $\dot{\mathbf{\vv x}} = A\mathbf{\vv x}$ ? Does there exist a function, call it $e^{At}$ , so that $\mathbf{x}(t) = e^{At}\mathbf{x}(0)$ ? How would we even begin to define such a thing?

Let’s do the “obvious” thing and start with the definition (PS), and replace the scalar $x$ with a matrix $X$ to obtain the matrix exponential of X:

e^X = I + X + \frac{X^2}{2!} + \frac{X^3}{3!} + \cdots = \sum_{k=0}^{\infty} \frac{X^k}{k!}, \quad (\text{MPS})

(2)

Although we can’t prove it yet, it can be shown that (MPS) converges for any $X$ , so this is a well defined object. Does (MPS) help with solving $\dot{\mathbf{\vv x}} = A\mathbf{\vv x}$ ? Let’s try the test solution $\mathbf{\vv x}(t) = e^{At}\mathbf{\vv x}(0)$ — this is exactly what we did for the scalar setting, but we replace $e^{at}$ with $e^{At}$ . Is this a solution to $\dot{\mathbf{\vv x}} = A\mathbf{\vv x}$ ?

First, we compute $A\mathbf{\vv x}(t) = Ae^{At}\mathbf{\vv x}(0)$ . Next, we need to compute $\frac{d}{dt}e^{At}\mathbf{\vv x}(0)$ . But how do we do this? We will rely on (MPS):

\begin{align*} \frac{d}{dt} e^{At} \vv x(0) &= \frac{d}{dt} \left(I + At + \frac{(At)^2}{2!} + \frac{(At)^3}{3!} + \cdots\right) \\ &= \frac{d}{dt}I + \frac{d}{dt}At + \frac{d}{dt}\frac{A^2t^2}{2!} + \frac{d}{dt}\frac{A^3t^3}{3!} + \cdots \\ &= 0 + A + A^2t + A^3\frac{t^2}{2} + \cdots \\ &= A + A^2t + \frac{A^3t^2}{2!} + \frac{A^4t^3}{3!} + \cdots \\ &= A\left(I + At + \frac{A^2t^2}{2!} + \frac{A^3t^3}{3!} + \cdots\right) \\ &= A e^{At} \vv x(0). \end{align*}

(3)

This worked, and we have found a general solution to $\dot{\vv x} = \vv Ax$ defined in terms of the matrix exponential!

This is very satisfying, as now our scalar and vector-valued problems have similar looking solutions defined in terms of appropriate exponential functions. The only thing that remains is to compute $e^{At}$ ! How do we do this? This is where all of the work we’ve done on diagonalization and Jordan forms really pays off!

4Computing the Matrix Exponential¶

4.1Case 1: Real eigenvalues, diagonalizable $A$ ¶

Suppose that $A \in \mathbb{R}^{n\times n}$ and has eigenvalues $\lambda_1, \lambda_2, \ldots, \lambda_n$ with corresponding linearly independent eigenvectors $\vv v_1, \vv v_2, \ldots,\vv v_n$ . Then we can write

A = V \Lambda V^{-1}, \text{ for } V = \bm \vv v_1, \vv v_2, \ldots, \vv v_n\em \text{ and } \Lambda = \text{diag}(\lambda_1, \lambda_2, \ldots, \lambda_n).

(4)

To compute $e^{At}$ we need to compute powers $(At)^k$ . Let’s work a few of these out using $A = V\Lambda V^{-1}$ :

\begin{align*} (At)^0 = I, \quad At = V\Lambda V^{-1}t, \quad A^2t^2 &= (V\Lambda V^{-1})(V\Lambda V^{-1})t^2, & A^3t^3 &= (V\Lambda V^{-1})A^2t^3 \\ &= V\Lambda^2 V^{-1}t^2 & &=(V\Lambda V^{-1})(V\Lambda^2 V^{-1})t^3 \\ & & &= V\Lambda^3 V^{-1}t^3 \end{align*}

(5)

There is a pattern: $(At)^k = V \Lambda^k V^{-1} t^k$ . This is nice, since computing powers of diagonal matrices is easy:

\Lambda^k = \begin{bmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{bmatrix}^k = \begin{bmatrix} \lambda_1^k & & \\ & \ddots & \\ & & \lambda_n^k \end{bmatrix}.

(6)

Let’s plug these expressions into (MPS):

\begin{align*} e^{At} &= I + At + \frac{A^2t^2}{2!} + \frac{A^3t^3}{3!} + \cdots \\ &= VV^{-1} + V\Lambda V^{-1}t + V\Lambda^2 V^{-1}\frac{t^2}{2!} + V\Lambda^3 V^{-1}\frac{t^3}{3!} + \cdots \\ &= V\left(I + \Lambda t + \frac{\Lambda^2 t^2}{2!} + \frac{\Lambda^3 t^3}{3!} + \cdots\right)V^{-1} \quad \text{(factor out } V(\cdot)V^{-1}\text{)} \\ &= V\left(\text{diag}\left(1+\lambda_1t+\frac{\lambda_1^2t^2}{2!}+\frac{\lambda_1^3t^3}{3!}, \ldots, 1+\lambda_nt+\frac{\lambda_n^2t^2}{2!}+\frac{\lambda_n^3t^3}{3!}\right)\right)V^{-1} \\ &= V \begin{bmatrix} e^{\lambda_1 t} & & \\ & \ddots & \\ & & e^{\lambda_n t} \end{bmatrix} V^{-1} \quad \text{(we recognize } 1+\lambda_i t+\frac{\lambda_i^2t^2}{2!}+\cdots \text{ as (PS))} \end{align*}

(7)

That’s very nice! We diagonalize $A$ , then exponentiate its eigenvalues to compute $e^{At}$ . Let’s plug this back in to $\vv x(t) = e^{At} \vv x(0)$ :

\vv x(t) = V \begin{bmatrix} e^{\lambda_1 t} & & \\ & \ddots & \\ & & e^{\lambda_n t} \end{bmatrix} V^{-1} \vv x(0).

(8)

Now, if we let $\vv c = V^{-1}\vv x(0)$ , we can write

\vv x(t) = \bm \vv v_1 \cdots \vv v_n\em \begin{bmatrix} e^{\lambda_1 t} & & \\ & \ddots & \\ & & e^{\lambda_n t} \end{bmatrix} \begin{bmatrix} c_1 \\ \vdots \\ c_n \end{bmatrix} = c_1 e^{\lambda_1 t}\vv v_1 + \cdots + c_n e^{\lambda_n t} \vv v_n,

(9)

recovering our previous solution, with the exact formula $\vv c = V^{-1} \vv x(0)$ we saw previously for the coefficients $c_1, \ldots, c_n$ !.

4.2Case 2: Imaginary Eigenvalues¶

We focus on the $2 \times 2$ case with ${A = \begin{bmatrix} 0 & \omega \\ -\omega & 0 \end{bmatrix} = \omega \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}}$ . In this case, we will compute the power series directly.

\begin{align*} A &= \omega \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}, & A^2 &= \omega^2 \begin{bmatrix} -1 & 0 \\ 0 & -1 \end{bmatrix}, & A^3 &= \omega^3 \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}, & A^4 &= \omega^4 \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \\ &= \omega J, & &= \omega^2 J^2, & &= \omega^3 J^3, & &= \omega^4 J^4 \\ A^5 &= \omega^5 J^5 = \omega^5 J, & A^6 &= \omega^6 J^6 = J^2, & A^7 &= \omega^7 J^7 = \omega^7 J^3, & A^8 &= \omega^8 J^8 = \omega^8 J^4, \end{align*}

(10)

etc. So putting this together in computing $e^{At}$ we get:

e^{At} = \begin{bmatrix} 1 - \frac{1}{2!}t^2 \omega^2 + \cdots & t \omega - \frac{1}{3!}t^3 \omega^3 + \cdots \\ -t \omega + \frac{1}{3!}t^3 \omega^3 + \cdots & 1 - \frac{1}{2!}t^2 \omega^2 + \cdots \end{bmatrix} = \begin{bmatrix} \cos \omega t & \sin \omega t \\ -\sin \omega t & \cos \omega t \end{bmatrix},

(11)

where we used the power series for $\sin \omega t$ and $\cos \omega t$ in the last equality. As expected, the matrix $A = \omega \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}$ has a matrix exponential which defines a rotation, at rate ω, so that

\vv x(t) = \begin{bmatrix} \cos \omega t & \sin \omega t \\ -\sin \omega t & \cos \omega t \end{bmatrix} \vv x(0).

(12)

4.3Case 3: Complex Eigenvalues¶

Let’s generalize our previous example to $A = \begin{bmatrix} 6 & \omega \\ -\omega & 6 \end{bmatrix}$ .The matrix $A$ has complex conjugate eigenvalues $\lambda_1 = 6 + i\omega$ and $\lambda_2 = 6 - i\omega$ . We will again compute the power series directly. To do so, we will use the following very useful fact:

Theorem 2

If $A$ and $B$ commute ( $AB = BA$ ), then $e^{A}e^B = e^{A+B}$ .

Proof 1 (Proof of Theorem 2)

Assume $A, B$ commute, i.e., $AB = BA$ . Expanding $e^{A}e^B$ using the power series definition, we have:

\begin{align*} e^Ae^B &= \left(\sum_{a = 0}^{\infty}{\frac{A^a}{a!}}\right)\left(\sum_{b =0}^{\infty}{\frac{B^b}{b!}}\right)\\ &= \sum_{a = 0}^{\infty}{\frac{A^a}{a!}\left(\sum_{b =0}^{\infty}{\frac{B^b}{b!}}\right)}\\ &= \sum_{p = 0}^{\infty}{\sum_{a=0}^{p}{ \frac{A^a}{a!}\cdot \frac{B^{p - a}}{(p-a)!} }} \end{align*}

(13)

The last equality in particular follows by grouping terms with the same values of $a + b$ . We also expand $e^{A + B}$ using the power series definition,

\begin{align*} e^{A + B} &= \sum_{p = 0}^{\infty}{\frac{(A + B)^p}{p!}}\\ &= \sum_{p = 0}^{\infty}{\sum_{a = 0}^{p}{\frac{\binom{p}{a}A^aB^{k - a}}{p!}}}\\ &= \sum_{p = 0}^{\infty}{\sum_{a = 0}^{p}{\frac{\left(\frac{p!}{a!(p - a)!}\right)A^aB^{p - a}}{p!}}}\\ &= \sum_{p = 0}^{\infty}{\sum_{a=0}^{p}{ \frac{A^a}{a!}\cdot \frac{B^{p - a}}{(p-a)!} }} \end{align*}

(14)

Here, the second equality follows from the binomial expansion and commutativity of $A, B$ . We see that the expressions for $e^Ae^B$ and $e^{A + B}$ are equal. Hence $AB = BA$ implies $e^{A}e^B = e^{A+B}$ .

We will strategically use this fact. First, defining $J = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}$ we note that we can write $A = 6I + \omega J$ .Importantly, $6I$ and $\omega J$ commute as $(6I)(\omega J) = (\omega J)(6I) = \omega 6J$ . Therefore,

e^{At} = e^{(6I + \omega J)t} = e^{6It} e^{\omega Jt} = \begin{bmatrix} e^{6t} & 0 \\ 0 & e^{6t} \end{bmatrix} \begin{bmatrix} \cos \omega t & \sin \omega t \\ -\sin \omega t & \cos \omega t \end{bmatrix} = e^{6t} \begin{bmatrix} \cos \omega t & \sin \omega t \\ -\sin \omega t & \cos \omega t \end{bmatrix}

(15)

4.4Case 4: Jordan Block¶

Assume $A = V \begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix} V^{-1}$ , for $V = \bm \vv v_1 & \vv v_2\em$ an eigenvector and generalized eigenvector of A.

Then following the same argument as in Case 1, we have that $e^{At} = V e^{\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}t} V^{-1}$ . To compute $e^{\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}t}$ , we note $\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}t = \lambda It + t\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}$ , and that these two terms commute. Hence: $e^{\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}t} = e^{\begin{bmatrix} \lambda & 0 \\ 0 & \lambda \end{bmatrix}t} e^{t\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}}$ . We note that

\begin{align*} e^{\begin{bmatrix} \lambda & 0 \\ 0 & \lambda \end{bmatrix}t} = \begin{bmatrix} e^{\lambda t} & 0 \\ 0 & e^{\lambda t} \end{bmatrix} \text{ and } e^{\begin{bmatrix} 0 & t \\ 0 & 0 \end{bmatrix}} &= \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} + \begin{bmatrix} 0 & t \\ 0 & 0 \end{bmatrix} \ \text{ (higher powers }=0) \\ &= \begin{bmatrix} 1 & t \\ 0 & 1 \end{bmatrix} \end{align*}

(16)

Allowing us to conclude that $e^{\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}t} = \begin{bmatrix} e^{\lambda t} & te^{\lambda t} \\ 0 & e^{\lambda t} \end{bmatrix}$ , and that

\begin{align*} \vv x(t) = e^{At} \vv x(0) &= \bm \vv v_1 & \vv v_2\em \begin{bmatrix} e^{\lambda t} & te^{\lambda t} \\ 0 & e^{\lambda t} \end{bmatrix} V^{-1} \vv x(0), \quad \text{and letting } \vv c = V^{-1}\vv x(0) \\ &= \bm \vv v_1 & \vv v_2\em \begin{bmatrix} c_1 e^{\lambda t} + c_2 te^{\lambda t} \\ c_2 e^{\lambda t} \end{bmatrix} = \left(c_1 e^{\lambda t} + c_2 te^{\lambda t}\right)\vv v_1 + c_2 e^{\lambda t} \vv v_2, \end{align*}

(17)

which we recognize from our previous section on Jordan Blocks.