Classes 4 · 5 · Tests 1 & 2

Vector Spaces
& Bases

A vector space is a set of vectors closed under addition and scalar multiplication. The column space, null space, and row space of a matrix are the key subspaces we work with throughout the course.

Vector Space Axioms

The 8 Axioms (summarized)

Addition: associative, commutative, has identity (⃗0), has inverses (−⃗v) Scalar multiplication: distributes over addition and addition over vectors, compatible (a(b⃗v) = (ab)⃗v), has identity (1⃗v = ⃗v)

Any set of objects satisfying these axioms is a vector space. In this course, we almost always use $\mathbb{R}^m$ (real column vectors with $m$ entries) as our vector space.

Matrix Subspaces

Column Space $\mathcal{C}(A)$

$$\mathcal{C}(A) = \{\vec{v} \in \mathbb{R}^m : \exists\, \vec{u} \in \mathbb{R}^n,\; A\vec{u} = \vec{v}\}$$

The set of all vectors reachable by the map $A : \mathbb{R}^n \to \mathbb{R}^m$. Dimension = rank of $A$. Also called the image or range of $A$. The system $A\vec{x} = \vec{b}$ is consistent if and only if $\vec{b} \in \mathcal{C}(A)$.

Null Space $\mathcal{N}(A)$

$$\mathcal{N}(A) = \{\vec{u} \in \mathbb{R}^n : A\vec{u} = \vec{0}\}$$

The set of all vectors mapped to zero. Dimension = $n - \text{rank}(A)$ (nullity). Found by solving $A\vec{u} = \vec{0}$ using RREF: free variables give basis vectors for the null space.

Rank-Nullity Theorem

$$\text{rank}(A) + \text{nullity}(A) = n \quad \text{(number of columns)}$$

For any $A \in \mathbb{R}^{m \times n}$: rank + dim(null space) = $n$. This is one of the most fundamental results in linear algebra.

Basis and Linear Independence

Linear Independence

Vectors {⃗u₁, ⃗u₂, ..., ⃗uₙ} are linearly independent if and only if: w₁⃗u₁ + w₂⃗u₂ + ··· + wₙ⃗uₙ = ⃗0 ⟹ w₁ = w₂ = ··· = wₙ = 0 Equivalently: no vector in the set is a linear combination of the others. Equivalently: [⃗u₁ ⃗u₂ ··· ⃗uₙ] has trivial null space.

A basis is a linearly independent spanning set. Any vector in the space has a unique representation as a linear combination of basis vectors.

Finding a Basis for the Column Space via RREF

1. Row reduce A to RREF 2. Identify the pivot columns in the RREF 3. Take those SAME COLUMN INDICES from the original matrix A The pivot columns of the ORIGINAL A form a basis for C(A). (Not the RREF columns — those are unit vectors, not the original data.)

Example: if RREF has pivots in columns 1 and 3, take columns 1 and 3 of the original $A$.

Orthogonality

Orthogonal Vectors and Subspaces

$$\vec{u} \perp \vec{v} \iff \vec{u} \cdot \vec{v} = \vec{u}^T\vec{v} = 0$$ Two subspaces S₁ and S₂ are orthogonal if every ⃗u ∈ S₁ and ⃗v ∈ S₂ satisfies ⃗u ⊥ ⃗v.

Fundamental orthogonality: the null space $\mathcal{N}(A)$ is orthogonal to the row space of $A$. The left null space of $A$ is orthogonal to the column space of $A$.

Orthonormal Basis & Orthogonal Matrix

Orthonormal basis {⃗q₁, ⃗q₂, ..., ⃗qₙ}: ‖⃗qⱼ‖ = 1 (unit length) ⃗qᵢᵀ⃗qⱼ = 0 (mutually orthogonal, i ≠ j) Assembled into matrix Q = [⃗q₁ ⃗q₂ ... ⃗qₙ]: QᵀQ = I and QQᵀ = I → Q⁻¹ = Qᵀ

An orthogonal matrix $Q$ preserves lengths and angles: $\|Q\vec{v}\| = \|\vec{v}\|$. MATLAB: orth(A) returns orthonormal basis for column space.

Practice Problems

W26 Test 1 · Q2 For $A = \begin{bmatrix}1&1&1\\1&2&3\\3&1&-1\end{bmatrix}$, find columns forming a basis for the column space. +
Worked Answer
Check if col3 is a linear combination of col1 and col2: solve $\begin{bmatrix}1\\1\\3\end{bmatrix}a + \begin{bmatrix}1\\2\\1\end{bmatrix}b = \begin{bmatrix}1\\3\\-1\end{bmatrix}$.

From row 1: $a + b = 1$. From row 2: $a + 2b = 3 \Rightarrow b = 2, a = -1$. Check row 3: $3(-1) + 1(2) = -1$ ✓.

So col3 = $-$col1 $+ 2\cdot$col2 → col3 is linearly dependent. Rank = 2.

Any two columns form a basis for the column space.
2019 Test 1 · Q6 $A = \begin{bmatrix}1&2&3\\-1&1&0\\2&1&3\end{bmatrix}$ has a column space of dimension 2. Find a vector NOT in the column space. +
Worked Answer
Test $\vec{b} = [-1,1,1]^T$: check if $A\vec{x} = \vec{b}$ is consistent. Augment $[A|\vec{b}]$ and row reduce:
$\begin{bmatrix}1&2&3&|-1\\-1&1&0&|1\\2&1&3&|1\end{bmatrix}$
$R_2 \leftarrow R_2 + R_1$: $\begin{bmatrix}1&2&3&|-1\\0&3&3&|0\\0&-3&-3&|3\end{bmatrix}$
$R_3 \leftarrow R_3 + R_2$: $\begin{bmatrix}1&2&3&|-1\\0&3&3&|0\\0&0&0&|3\end{bmatrix}$

Last row gives $0 = 3$ — inconsistent. $[-1,1,1]^T$ is NOT in the column space.
Conceptual A matrix $A \in \mathbb{R}^{4\times 6}$ has rank 3. What are the dimensions of its column space and null space? +
Worked Answer
Column space dimension = rank = 3.
Null space dimension = $n - \text{rank} = 6 - 3 = $ 3.

Verify rank-nullity: $3 + 3 = 6 = n$ ✓.