2.6 Adjoint Systems, Left Null Space, and Row Space

1Reading¶

Material related to this page, as well as additional exercises, can be found in ALA Ch. 2.5.

2Learning Objectives¶

By the end of this page, you should know:

the transpose of a matrix
Adjoint systems
Left Null Space
Row Space
the fundamental theorem of linear algebra and its implication

3Matrix Transpose¶

The transpose $A^{\top}$ of an $m \times n$ matrix $A$ is the $n \times m$ matrix obtained by interchanging its rows and columns. Hence, if ${B = A^{\top}}$ , then $b_{ij} = a_{ji}$ .

Properties

$\left(A^{\top}\right)^{\top} = A$ (Transpose of transpose brings back to where you started)
$(A + B)^{\top} = A^{\top} + B^{\top}$ (Transpose and addition commute)
$(AB)^{\top} = B^{\top}A^{\top}$ (Reverses order on products)
$\left(A^{-1}\right)^{\top} = \left(A^{\top}\right)^{-1} = A^{-T}$ (inverse and transpose commute, provided $A^{-1}$ exists)

A special case is the product of a row vector $\vv v^{\top}$ and a column vector $\vv w$ (which we will revisit):

\vv v^{\top} \vv w = \left(\vv v^{\top} \vv w\right)^{\top} = \vv w^{\top} \vv v,

(1)

because the product in (1) is a scalar and the transpose of a scalar is itself.

3.1Python Break!¶

Transpose of a NumPy array A is given by A.T. Below are examples of matrix (and vector) transposes using NumPy.

## Transpose of NumPy arrays
import numpy as np

A = np.array([[1, 2, 3],
              [4, 5, 6]])
AT = A.T # Transpose of the A matrix

c = np.array([[1],
              [2],
              [3]]) # column vector
cT = c.T # transpose of a column vector

c1 = np.array([1, 2, 3]) # ?-vector
c1T = c1.T

print("Tranpose of \nA: \n", AT, "\nc: \n", cT, "\nc1: \n", c1T)

Tranpose of 
A: 
 [[1 4]
 [2 5]
 [3 6]] 
c: 
 [[1 2 3]] 
c1: 
 [1 2 3]

4Adjoint System¶

This section explores the properties of the system of linear equations defined by $A^{\top}$ , rather than $A$ . This adjoint system might first appear as some abstract nonsense that only mathematicians care about, but we will show that is has very practical consequences and interpretations.

At first glance, the solutions to $A \vv x = \vv b$ and the solutions to its adjoint $A^{\top} \vv y = \vv f$ seem unrelated. We will see some very surprising connections between them that will be revisited in even greater depth later in the course.

5Row Space and Left Null Space¶

We start by introducing the last two fundamental subspaces associated with a matrix $A$ .

It is called the row space because it is the subspace of $\mathbb{R}^n$ spanned by the rows of $A$ (more precisely, by the columns obtained from transposing the rows of $A$ ).

It is called the left null space of $A$ because up to taking a transpose, LNull $(A)$ is composed of row vectors $\vv w^{\top}$ satisfying $\vv w^{\top} A = \vv 0^{\top}$ .

Example 3 (Finding the kernel, image, cokernel, and coimage of a

3\times 4

matrix)

Consider the $3\times 4$ matrix:

\begin{align*} A = \bm 1 & 5 & -2 & 2 \\ 0 & 1 & 2 & -5 \\ 1 & 3 & -3 & 7 \em \end{align*}

(8)

Let’s find the four fundamental subspaces (i.e., give bases for these subspaces) of $A$ .

Finding the nullspace/kernel. The nullspace is the solution to the homogenous system:

\begin{align*} A \vv x = \bm 1 & 5 & -2 & 2 \\ 0 & 1 & 2 & -5 \\ 1 & 3 & -3 & 7 \em \vv x = \vv 0. \end{align*}

(9)

We apply the usual method of Gauss-Jordan elimination to find a reduced row echelon form of $A$ , given by the equivalent augmented matrix:

\begin{align*} \left[\begin{array}{cccc|c} 1 & 0 & 0 & 7 & 0 \\ 0 & 1 & 0 & -5/3 & 0 \\ 0 & 0 & 1 & -5/3 & 0 \end{array}\right]. \end{align*}

(10)

which has basic variables $\vv{x_1}, \vv{x_2}, \vv{x_3}$ and free variable $\vv{x_4}$ . The number of free variables, 1, is the dimension of the kernel. Solving this, we get:

\begin{align*} \vv{x_1} = -7 \vv{x_4}, \quad \vv{x_2} = \frac 5 3 \vv{x_4}, \quad \vv{x_3} = \frac 5 3 \vv{x_4}. \end{align*}

(11)

meaning that a basis for the kernel is given by $\left\{ \bm -7 \\ 5/3 \\ 5/3 \\ 1 \em \right\}$ .

Finding the columnspace/image. To find a basis for the image of $A$ , we perform Gaussian elimination on $A^\top = \left[\begin{array}{ccc} 1 & 0 & 1 \\ 5 & 1 & 3 \\ -2 & 2 & -3\\ 2 & -5 & 7 \end{array}\right]$ . To gain an initial intuition for this procedure, note that performing row operations on $A^\top$ is equivalent to performing column operations, which preserve the columnspace of $A$ , on $A$ . We get the equivalent matrix:

\begin{align*} \bm 1 & 0 & 1 \\ 0 & 1 & -2 \\ 0 & 0 & 3\\ 0 & 0 & 0 \em. \end{align*}

(12)

A basis for $\text{Col}(A)$ is given by $\left\{ \bm 1\\0\\1\em, \bm 0\\1\\-2\em, \bm 0\\0\\3\em\right\}$ . To see why, we know that:

\begin{align*} \text{span} \left\{ \bm 1\\0\\1\em, \bm 0\\1\\-2\em, \bm 0\\0\\3\em \right\} = \text{Col}(A) \end{align*}

(13)

simply because column operations on $A$ preserve its columnspace. Furthermore, its clear that these three vectors are linearly independent. These two facts together imply that this set is a basis for $\text{Col}(A)$ .

Finding the left nullspace/cokernel. The left nullspace is the solution to the homogenous system:

\begin{align*} A^\top \vv y = \bm 1 & 0 & 1 \\ 5 & 1 & 3 \\ -2 & 2 & -3\\ 2 & -5 & 7 \em \vv y = \vv 0. \end{align*}

(14)

We apply the usual method of Gauss-Jordan elimination to find a reduced row echelon form of the augmented matrix, given by the equivalent augmented matrix:

\begin{align*} \left[\begin{array}{ccc|c} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 \end{array}\right]. \end{align*}

(15)

which has basic variables $\vv{y_1}, \vv{y_2}, \vv{y_3}$ , and no free variables, meaning that the left nullspace of $A$ is trivial, $\text{LNull}(A) = \{\vv 0\}$ .

Finding the rowspace/coimage. To find a basis for the rowspace of $A$ , we perform Gaussian elimination on $A$ , and get the equivalent matrix:

\begin{align*} \bm 1 & 5 & -2 & 2 \\ 0 & 1 & 2 & -5 \\ 0 & 0 & 3 & -5 \em. \end{align*}

(16)

And hence a basis for $\text{Row}(A)$ is given by $\left\{ \bm 1\\5\\-2\\2\em, \bm 0\\1\\2\\-5\em, \bm 0\\0\\3\\-5\em\right\}$ .

5.1Python break!¶

In Example 3, we found the four fundamental subspacesthe using hand. In Python, we can use the sympy package (it’s also straightforward to do this in numpy) to do the same.

## Find the four fundamental subspaces of a matrix
from sympy import *

A = Matrix([[1, 5, -2, 2],
            [0, 1, 2, -5],
            [1, 3, -3, 7]])

print("The A matrix:\n", A)

A_Null = A.nullspace()
A_col = A.columnspace()

print("Null space of A: \n", A_Null, "\nColumn space of A: \n", A_col)

AT = A.T

A_LNull = AT.nullspace()
A_row = AT.columnspace()

print("Left Null space of A: \n", A_LNull, "\nRow space of A: \n", A_row)

The A matrix:
 Matrix([[1, 5, -2, 2], [0, 1, 2, -5], [1, 3, -3, 7]])
Null space of A: 
 [Matrix([
[ -7],
[5/3],
[5/3],
[  1]])] 
Column space of A: 
 [Matrix([
[1],
[0],
[1]]), Matrix([
[5],
[1],
[3]]), Matrix([
[-2],
[ 2],
[-3]])]
Left Null space of A: 
 [] 
Row space of A: 
 [Matrix([
[ 1],
[ 5],
[-2],
[ 2]]), Matrix([
[ 0],
[ 1],
[ 2],
[-5]]), Matrix([
[ 1],
[ 3],
[-3],
[ 7]])]

6The Fundamental Theorem¶

The proof of Theorem 1 can be found in pp. 114-118 of ALA. We will focus on some remarkable implications of Theorem 1 which are given below.

The statement in Theorem 2 holds true with Col $(A) \to$ Row $(A)$ and Null $(A) \to$ LNull $(A)$ .

6.1Python break!¶

We verify Theorem 2 below in Python using the fundamental subspaces we computed previously.

# Verifying Theorem-1

dim_col = len(A_col)
rank = A.rank()
dim_null = len(A_Null)
n_cols = A.shape[1] # what is A.shape[0]?

print("Rank: ", rank, ". Dimension of column space is ", dim_col, "and null space is ", dim_null, 
      "\nNumber of columns n is ", n_cols)

Rank:  3 . Dimension of column space is  3 and null space is  1 
Number of columns n is  4