Matrices

Matrix

A matrix is a collection of numbers that can be indexed in a particular way, this indexing mimics a two dimensional grid. The dimensions of a matrix are stated as $r \times c$ where $r, c \in ℕ_{1}$ when it has $m$ rows and $n$ columns.

Suppose that $A$ is a matrix, then we can access the value at the $i$ -th row and $j$ -th column by writing $A_{i, j}$ , we can graphically represent this information by introducing variables for the coefficients and writing them like this:

A = [\begin{matrix} a_{1, 1} & a_{1, 2} & \dots & a_{1, n} \\ a_{2, 1} & a_{2, 2} & \dots & a_{2, n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m, 1} & a_{m, 2} & \dots & a_{m, n} \end{matrix}]

Matrices are denoted by capital letters such as $A, B, M, N$ .

Set of Matrices

Suppose that

r, c \in ℕ_{1}

then we denote the set of all

r \times c

matrices with cofficients from the set

T

M_{r \times c} (T)

Row Matrix

Any matrix

R \in M_{1 \times c} (T)

for some

c \in ℕ_{1}

is known as a row matrix.

Row of a Matrix

Suppose that $A$ is a matrix, then if we want to extract the $i - t h$ row of $A$ we can write $A_{i, -}$ .

Note that this is a function of the form $M_{r \times c} (T) \to M_{1 \times c} (T)$

The notation

A_{i, -}

intends to mimic freezing at a certain row and then sweeping over all columns.

Row Matrix Representation

Given a matrix

A

, there is an associated row matrix representation, of the following form

A = [\begin{matrix} \leftarrow & A_{1, -} & \to \\ \leftarrow & A_{2, -} & \to \\ \leftarrow & ⋮ & \to \\ \leftarrow & A_{m, -} & \to \end{matrix}]

Where the arrows are there to remind you that they expand horizontally. Also observe that this matrix is an element of $M_{r \times 1} (M_{1 \times c} (ℝ))$

Column Matrix

Any matrix

C \in M_{r \times 1} (T)

for some

r \in ℕ_{1}

is known as a row matrix.

We will denote column or row matrices with a lower case boldfaced character, like $𝐱$

Column of a Matrix

Suppose that $A$ is a matrix, then if we want to extract the $j$ -th row of $A$ we can write $A_{|, j}$ .

Note that this is a function of the form $M_{r \times c} (T) \to M_{r \times 1} (T)$

The

|

denotes that the row variable may be varied vertically, while the column index is fixed in place. If we want to represent the matrix as columns graphically, we may do so like this

Column Matrix Representation

Given a matrix

A

, there is an associated column matrix representation, of the following form

A = [\begin{matrix} ↑ & ↑ & ↑ & ↑ \\ A_{|, 1} & A_{|, 2} & \dots & A_{|, n} \\ ↓ & ↓ & ↓ & ↓ \end{matrix}]

Where the arrows are there to remind you that they expand graphically expand vertically. Also observe that this matrix is an element of $M_{1 \times c} (M_{r \times 1} (ℝ))$ .

We can connect some of the previous definitions here to say that

R

is a column matrix whose entries are row matrices.

Matrix Multiplication

Suppose $r_{A}, c_{A}, r_{B}, c_{B} \in ℕ_{1}$ such that $c_{A} = r_{B}$ and that we have two matrices $A, B \in M_{r_{A} \times c_{A}} (ℝ), M_{r_{B} \times c_{B}} (ℝ)$ , defined as

A = [\begin{matrix} a_{1, 1} & a_{1, 2} & \dots & a_{1, c_{A}} \\ a_{2, 1} & a_{2, 2} & \dots & a_{2, c_{A}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{r_{A}, 1} & a_{r_{A}, 2} & \dots & a_{r_{A}, c_{A}} \end{matrix}], B = [\begin{matrix} b_{1, 1} & b_{1, 2} & \dots & b_{1, c_{B}} \\ b_{2, 1} & b_{2, 2} & \dots & b_{2, c_{B}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ b_{r_{B}, 1} & b_{r_{B}, 2} & \dots & b_{r_{B}, c_{B}} \end{matrix}]

Then we define the matrix multiplication of $A$ and $B$ as the matrix $P \in M_{r_{A} \times c_{B}}$

P = [\begin{matrix} c_{1, 1} & c_{1, 2} & \dots & c_{1, c_{P}} \\ c_{2, 1} & c_{2, 2} & \dots & c_{2, c_{P}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ c_{r_{P}, 1} & c_{r_{P}, 2} & \dots & c_{r_{P}, c_{P}} \end{matrix}]

Where:

c_{i, j} : = a_{i, 1} b_{1, j} + a_{i, 2} b_{2, j} + \dots + a_{i, c_{A}} b_{c_{A}, j} = \sum_{k = 1}^{c_{A}} a_{i, k} b_{k, j}

Matrix Addition and Scalar Multiplication

Let

A, B \in M_{r \times c} (ℝ)

and

λ \in ℝ

. We define:

{(A + B)}_{i, j} : = A_{i, j} + B_{i, j}, {(λ A)}_{i, j} : = λ A_{i, j} .

We also define

A - B : = A + (- 1) B

Identity Matrix

For

n \in ℕ_{1}

, the

n \times n

identity matrix is:

{(I_{n})}_{i, j} : = {\begin{matrix} 1 & i = j \\ 0 & i \neq j . \end{matrix}

Matrix Transpose

Let

A \in M_{r \times c} (ℝ)

. The transpose of

A

is the matrix

A^{T} \in M_{c \times r} (ℝ)

defined by:

{(A^{T})}_{i, j} : = A_{j, i} .

Reflection Matrix About a Line Through the Origin

Let

θ \in ℝ

, and let

L_{θ} \subseteq ℝ^{2}

be the line through the origin in the direction

(\cos (θ), \sin (θ))

. The matrix which reflects vectors across

L_{θ}

is:

[\begin{matrix} \cos (2 θ) & \sin (2 θ) \\ \sin (2 θ) & - \cos (2 θ) \end{matrix}] .

Let $v = (x, y) \in ℝ^{2}$ , and write $v$ in polar form: $v = (x, y) = (r \cos (α), r \sin (α)) .$ Suppose $v^{'}$ is the reflection of $v$ across the line $L_{θ}$ , and write: $v^{'} = (r \cos (γ), r \sin (γ)) .$

Let $ϕ = θ - α$ . Since reflection across $L_{θ}$ places $v^{'}$ on the other side of the line by the same angle, $γ = θ + ϕ$ , and hence: $γ = θ + (θ - α) = 2 θ - α .$ Therefore: $v^{'} = (r \cos (2 θ - α), r \sin (2 θ - α)) .$

Using the trigonometric angle addition formulas: $\begin{matrix} v^{'} & = (r \cos (2 θ + (- α)), r \sin (2 θ + (- α))) \\ = (r (\cos (2 θ) \cos (- α) - \sin (2 θ) \sin (- α)), r (\sin (2 θ) \cos (- α) + \cos (2 θ) \sin (- α))) . \end{matrix}$ By the fact that sine is odd and cosine is even: $\begin{matrix} v^{'} & = (r (\cos (2 θ) \cos (α) + \sin (2 θ) \sin (α)), r (\sin (2 θ) \cos (α) - \cos (2 θ) \sin (α))) \\ = (\cos (2 θ) r \cos (α) + \sin (2 θ) r \sin (α), \sin (2 θ) r \cos (α) - \cos (2 θ) r \sin (α)) . \end{matrix}$

Since $x = r \cos (α)$ and $y = r \sin (α)$ , we have: $v^{'} = (\cos (2 θ) x + \sin (2 θ) y, \sin (2 θ) x - \cos (2 θ) y) .$ Equivalently, using matrix multiplication: $[\begin{matrix} x^{'} \\ y^{'} \end{matrix}] = [\begin{matrix} \cos (2 θ) & \sin (2 θ) \\ \sin (2 θ) & - \cos (2 θ) \end{matrix}] [\begin{matrix} x \\ y \end{matrix}] .$ Therefore the reflection matrix about $L_{θ}$ is: $[\begin{matrix} \cos (2 θ) & \sin (2 θ) \\ \sin (2 θ) & - \cos (2 θ) \end{matrix}] .$

Two Dimensional Rotation Matrix

For

θ \in ℝ

, the matrix which rotates vectors in

ℝ^{2}

counterclockwise by angle

θ

is:

R_{θ} = [\begin{matrix} \cos (θ) & - \sin (θ) \\ \sin (θ) & \cos (θ) \end{matrix}] .

Composition of Two Reflections is a Rotation

Let

L_{α}

and

L_{β}

be lines through the origin in

ℝ^{2}

, where

L_{α}

has direction

(\cos (α), \sin (α))

L_{β}

has direction

(\cos (β), \sin (β))

, and

θ = β - α

. Reflecting first across

L_{α}

, and then across

L_{β}

, is the same as rotating by

2 θ

S_{β} S_{α} = R_{2 θ} .

By the reflection matrix formula, the reflection matrices across $L_{α}$ and $L_{β}$ are: $S_{α} = [\begin{matrix} \cos (2 α) & \sin (2 α) \\ \sin (2 α) & - \cos (2 α) \end{matrix}], S_{β} = [\begin{matrix} \cos (2 β) & \sin (2 β) \\ \sin (2 β) & - \cos (2 β) \end{matrix}] .$

Using matrix multiplication: $\begin{matrix} S_{β} S_{α} & = [\begin{matrix} \cos (2 β) & \sin (2 β) \\ \sin (2 β) & - \cos (2 β) \end{matrix}] [\begin{matrix} \cos (2 α) & \sin (2 α) \\ \sin (2 α) & - \cos (2 α) \end{matrix}] \\ = [\begin{matrix} \cos (2 β) \cos (2 α) + \sin (2 β) \sin (2 α) & \cos (2 β) \sin (2 α) - \sin (2 β) \cos (2 α) \\ \sin (2 β) \cos (2 α) - \cos (2 β) \sin (2 α) & \sin (2 β) \sin (2 α) + \cos (2 β) \cos (2 α) \end{matrix}] . \end{matrix}$

By the trigonometric angle addition formulas, this simplifies to: $\begin{matrix} S_{β} S_{α} & = [\begin{matrix} \cos (2 β - 2 α) & - \sin (2 β - 2 α) \\ \sin (2 β - 2 α) & \cos (2 β - 2 α) \end{matrix}] \\ = [\begin{matrix} \cos (2 θ) & - \sin (2 θ) \\ \sin (2 θ) & \cos (2 θ) \end{matrix}] . \end{matrix}$ By the definition of the two dimensional rotation matrix, this is $R_{2 θ}$ .

Composition of Two Plane Reflections is a Rotation

Let

P_{1}, P_{2} \subseteq ℝ^{3}

be planes through the origin, and suppose that

A : = P_{1} \cap P_{2}

is a line. Let

A^{⟂}

be the plane perpendicular to

A

. Then

P_{1} \cap A^{⟂}

and

P_{2} \cap A^{⟂}

are lines through the origin in

A^{⟂}

. If the oriented angle from

P_{1} \cap A^{⟂}

P_{2} \cap A^{⟂}

θ

, then reflecting first across

P_{1}

, and then across

P_{2}

, is the same as rotating around the axis

A

2 θ

Every vector $x \in ℝ^{3}$ can be written uniquely as $x = x_{A} + x_{⟂},$ where $x_{A} \in A$ and $x_{⟂} \in A^{⟂}$ .

Since $A \subseteq P_{1}$ and $A \subseteq P_{2}$ , reflection across either plane fixes every vector in $A$ . Thus the $x_{A}$ component is unchanged by both reflections.

It remains to understand what happens to $x_{⟂}$ . Inside the plane $A^{⟂}$ , the intersections $L_{1} : = P_{1} \cap A^{⟂}, L_{2} : = P_{2} \cap A^{⟂}$ are lines through the origin. Reflection across $P_{1}$ restricts on $A^{⟂}$ to reflection across $L_{1}$ , and reflection across $P_{2}$ restricts on $A^{⟂}$ to reflection across $L_{2}$ .

By the two-dimensional composition of reflections theorem, reflecting first across $L_{1}$ , and then across $L_{2}$ , rotates $A^{⟂}$ by $2 θ$ . Therefore the composition fixes the component along $A$ and rotates the perpendicular component by $2 θ$ , which is exactly rotation around the axis $A$ by $2 θ$ .

Vector as a Column Matrix

For each

n \in ℕ_{1}

, define a function

to_col : ℝ^{n} \to M_{n \times 1} (ℝ)

by:

to_col ((a_{1}, \dots, a_{n})) : = [\begin{matrix} a_{1} \\ ⋮ \\ a_{n} \end{matrix}] .

Vector as a Row Matrix

For each

n \in ℕ_{1}

, define a function

to_row : ℝ^{n} \to M_{1 \times n} (ℝ)

by:

to_row ((a_{1}, \dots, a_{n})) : = [\begin{matrix} a_{1} & \dots & a_{n} \end{matrix}] .

The notation $to_col$ and $to_row$ is overloaded across dimensions: if $x \in ℝ^{n}$ , then the dimension $n$ determines which function is being used and what size matrix is produced.

Matrix Functions

Suppose that

𝐌 \in M_{r, c} (T)

and that

f : T \to S

is a function, then we augment this function to

𝐟 : M_{r, c} (T) \to M_{r, c} (S)

as follows:

𝐟 ([\begin{matrix} m_{1, 1} & m_{1, 2} & \dots & m_{1, c_{M}} \\ m_{2, 1} & m_{2, 2} & \dots & m_{2, c_{M}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ m_{r_{M}, 1} & m_{r_{M}, 2} & \dots & m_{r_{M}, c_{M}} \end{matrix}]) = [\begin{matrix} f (m_{1, 1}) & f (m_{1, 2}) & \dots & f (m_{1, c_{M}}) \\ f (m_{2, 1}) & f (m_{2, 2}) & \dots & f (m_{2, c_{M}}) \\ ⋮ & ⋮ & ⋱ & ⋮ \\ f (m_{r_{M}, 1}) & f (m_{r_{M}, 2}) & \dots & f (m_{r_{M}, c_{M}}) \end{matrix}]

Packing Function

Suppose that

x \in ℝ

, then we define the following function

pack : ℝ \to M_{1 \times 1} (ℝ)

pack (x) : = [\begin{matrix} x \end{matrix}]

Unpacking Function

We define

unpack : M_{1 \times 1} (ℝ) \to ℝ

by:

unpack ([\begin{matrix} x \end{matrix}]) : = x .

The Packing and Unpacking Functions are Inverses

TODO

One by one Matrices are Isomorphic to R

ℝ

and

M_{1, 1} (ℝ)

are isomorphic.

Row and Column Matrices are Isomorphic to

ℝ^{n}

Suppose that

n \in ℕ_{1}

, then

M_{1, n} (ℝ)

and

M_{n, 1} (ℝ)

are isomorphic to

ℝ^{n}

Nested Matrix Multiplication Unpacking

Given two matrices $A \in M_{a \times b} (ℝ) B \in M_{c \times d} (ℝ)$ then product is equal to

unpack ([\begin{matrix} A_{1, -} \cdot B_{|, 1} & A_{1, -} \cdot B_{|, 2} & \dots & A_{1, -} \cdot B_{|, d} \\ A_{2, -} \cdot B_{|, 1} & A_{2, -} \cdot B_{|, 2} & \dots & A_{2, -} \cdot B_{|, d} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ A_{a, -} \cdot B_{|, 1} & A_{a, -} \cdot B_{|, 2} & \dots & A_{a, -} \cdot B_{|, d} \end{matrix}])

With $A_{i, -}$ $B_{|, j}$ and $unpack$ is being used as a matrix function

Matrix Multiplication with Rows and Columns

Suppose that $A, B$ are matrices, and that $A_{R}, B_{C}$ are their row matrix and column matrix representations, then

unpack (A_{R} B_{C}) = A B

where

unpack

is being used as a matrix function

column matrix multiplication

suppose that

C = [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{k} \end{matrix}]

is a column matrix and

M \in M_{n \times k} (ℝ)

, then

M C = \sum_{i = 1}^{k} c_{i} M_{|}, i} \in M_{n \times 1} (ℝ)

By the definition of matrix multiplication the answer should be equal to

[\begin{matrix} m_{1, 1} & m_{1, 2} & \dots & m_{1, k} \\ m_{2, 1} & m_{2, 2,} & \dots & m_{2, k} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ m_{j, 1} & m_{j, 2} & \dots & m_{j, k} \end{matrix}]

[\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{k} \end{matrix}]

= [\begin{matrix} c_{1} \cdot m_{1, 1} + c_{2} \cdot m_{1, 2} + \dots + c_{k} \cdot m_{1, k} \\ c_{1} \cdot m_{2, 1} + c_{2} \cdot m_{2, 2} + \dots + c_{k} \cdot m_{2, k} \\ ⋮ \\ c_{1} \cdot m_{j, 1} + c_{2} \cdot m_{j, 2} + \dots + c_{k} \cdot m_{j, k} \end{matrix}]

= c_{1} M_{|}, 1} + c_{2} M_{|}, 2} + \dots + c_{k} M_{|}, k} \in M_{n \times 1} (ℝ)

unit column matrix

e_{i}

is a column matrix who's i-th row is

1

and the rest are

0

column extraction

suppose

M \in M_{j \times k}

then the

n

-th column of

M

is given by

M

multiplied by

e_{n}

(where

1 \leq n \leq k

)

Due to the way that column matrix multiplication works, we can see that $M e_{n} = \sum_{i = 0}^{k} {(e_{n})}_{i} M_{|}, i} = M_{|}, n}$ this is clear since only the $n$ -th component of $e_{n}$ is $1$ and the rest are $0$ .

Complex Transpose

We denote the complex transpose of a matrix as

A^{*} : = {\overline{A}}^{T}

Hermitian Matrix

A matrix

A \in M_{n \times n} (ℂ)

is said to be hermitian diff

A = A^{*}

where we've applied the complex-transpose

Note that the a Hermitian Matrix is also called symmetric or self-adjoint.

Normal

We say that a matrix is normal if

A^{*} A = A A^{*}