Consider an $m\times n$ matrix

$$A=\begin{pmatrix}a_{11} & \cdots & a_{1n}\\\vdots & & \vdots\\a_{m1} & \cdots & a_{mn}\end{pmatrix}.$$

The columns of $A$ generate a vector space, which is a subspace of $\mathbb{R}^m$, called the column space of $A$. The dimension of the subspace is called the column rank of $A$. Similarly the rows of $A$ generate a subspace of $\mathbb{R}^n$, called the row space of $A$ and the dimension of this subspace is called the row rank of $A$. It turns out that the column rank and the row rank must be equal. So, we simply call the column rank or the row rank of $A$, the rank of $A$.

There are a couple important theorems regarding the rank of a matrix. They are introduced without proofs.

Theorem. Row and column operations do not change the row rank of a matrix, nor do they change the column rank.

Remark. Row and column operations only change basis of row space or column space.

Theorem. Let $A$ be a matrix of rank $r$. By a succession of row and column operations, the matrix can be transformed to the matrix having components equal to $1$ on the diagonal of the first $r$ rows and columns, and $0$ everywhere else.

$$\begin{pmatrix}1 & 0 & \cdots & 0 & 0 &\cdots &0\\0 & 1 & \cdots & 0 & 0 & \cdots &0\\\vdots & &\ddots &\vdots&\vdots& &\vdots\\0 & 0 &\cdots & 1& 0 &\cdots &0\\0 & 0 &\cdots & 0& 0 &\cdots &0\\\vdots & & &\vdots&\vdots&\ddots&\vdots\\0 & 0 &\cdots & 0& 0 &\cdots &0\end{pmatrix}$$

Example. Find the rank of the matrix $\begin{pmatrix}2 & 1 & 1\\0 & 1 & -1\end{pmatrix}$.

Solution. There are only two rows, so the rank will be at most 2. On the other hand, the column vectors $\begin{pmatrix}2\\0\end{pmatrix}$ and $\begin{pmatrix}1\\1\end{pmatrix}$ are linear independent. Therefore, the rank is 2.

Example. Find the rank of the matrix

$$\begin{pmatrix}1 & 2 & -3\\2 & 1 & 0\\-2 & -1 & 3\\-1 & 4 & -2\end{pmatrix}.$$

Solution. Since there are three columns, the rank will be at most 3. Subtract 2 times column 1 from column 2; add 3 times column 1 to column 3. The resulting matrix is

$$\begin{pmatrix}1 & 0 & 0\\2 & -3 & 6\\-2 & 3 & -3\\-1 & 6 & -5\end{pmatrix}.$$

Add 2 times column 2 to column 3. The resulting matrix is

$$\begin{pmatrix}1 & 0 & 0\\2 & -3 & 0\\-2 & 3 & 3\\-1 & 6 & 7\end{pmatrix}.$$

This matrix is in column echelon form and one can easily see that the first three rwo vectors are linearly independent. Therefore, the rank is 3.

Let $V$ be a vector space. $v_1,\cdots,v_n\in V$ are said to be linearly dependent if there exist numbers $a_1,\cdots,a_n$ not all equal to $0$ such that
$$a_1v_1+\cdots+a_n=O.$$
If there do not exist such numbers, then we say $v_1,\cdots,v_n$ are linearly independent. That is, $v_1,\cdots,v_n$ are linearly independent if whenever $a_1v_1+\cdots+a_nv_n=O$, $a_1=\cdots=a_n=0$.

Example. In $\mathbb{R}^n$, the standard unit vectors $E_1,\cdots,E_n$ are linearly independent.

Example. The vectors $(1,1)$ and $(-3,2)$ are linearly indepdent in $\mathbb{R}^2$.

A set of vectors $\{v_1,\cdots,v_n\}\subset V$ is said to be a basis of $V$ if $v_1,\cdots,v_n$ generate $V$ and that they are linearly independent.

Example. The vectors $E_1,\cdots,E_n$ form a basis of $\mathbb{R}^n$.

Example. The vectors $(1,1)$ and $(-1,2)$ form a basis of $\mathbb{R}^2$.

In general for $\mathbb{R}$, the following theorem holds.

Theorem. Let $(a,b)$ and $(c,d)$ be two vectors in $\mathbb{R}^2$.

(i) They are linearly depedendent if and only if $ad-bc=0$.

(ii) If they are  indepdendent, they form a basis of $\mathbb{R}^2$.

Proof. Exercise

Let $V$ be a vector space and let $\{v_1,\cdots,v_n\}$ be a basis of $V$. If $v\in V$ is written as a linear combination
$$v=x_1v_1+\cdots+x_nv_n,$$
$(x_1,\cdots,x_n)$ is called the coordinates of $v$ with respect to the basis $\{v_1,\cdots,v_n\}$. For each $i=1,\cdots,n$, $x_i$ is called the i-th coordinate. The following theorem says that there can be only one set of coordinates for a given vector.

Theorem. Let $V$ be a vector space. Let $v_1,\cdots,v_n$ be linearly independent elements of $V$. If $x_1,\cdots,x_n$ and $y_1,\cdots,y_n$ are numbers such that
$$x_1v_1+\cdots+x_nv_n=y_1v_1+\cdots+y_nv_n,$$
then $x_i=y_i$ for all $i=1,\cdots,n$.

Proof. It is strightforward from the definition of linearly independent vectors.

Example. Find the coordinates of $(1,0)$ with respect to the two vectors $(1,1)$ and $(-1,2)$.

Example. The two functions $e^t$ and $e^{2t}$ are linearly independent.

Proof. Exercise.

Theorem. Let $v,w$ be two vectors of a vector space $V$. They are linearly dependent if and only if one of them is a scalr multiple of the other, i.e there is a number $c$ such that $v=cw$ or $w=cv$.

Proof. Exercise.

If one basis of a vector space $V$ has $n$ elements and another basis has $m$ elements, then $n=m$. The number of elements in any basis of a vector space $V$ is called the dimension of $V$ and is denoted by $\dim V$.

Let $V$ be a vector space and let $v_1,\cdots, v_n\in V$. $V$ is said to be generated by $v_1,\cdots,v_n$ if given an element $v\in V$, there exist numbers $x_1,\cdots, x_n$ such that
$$v=x_1v_1+\cdots+x_nv_n.$$
The expression $x_1v_1+\cdots+x_nv_n$ is called a linear combination of $v_1,\cdots,v_n$. The numbers $x_1,\cdots,x_n$ are called the coefficients of the linear combination.

Example. Let $E_1,\cdots,E_n$ be the standard unit vectors in $\mathbb{R}^n$. Then $E_1,\cdots,E-n$ generate $\mathbb{R}^n$.

Proof. Gievn $X=(x_1,\cdots,x_n)\in\mathbb{R}^n$,
$$X=\sum_{i=1}^nx_iE_i.$$

Proposition. The set of all linear combinations of $v_1,\cdots,v_n$ is a subspace of $V$.

Proof. Straightforward.

Example. Let $v_1$ be a non-zero element of a vector space $V$, and let $w$ be any element of $V$. The set
$$\{w+tv_1: t\in\mathbb{R}\}$$
is the line passing through $w$ in the direction of $v_1$. This line is not a subspace, however if $w=O$, it is a subspace of $V$, generated by a single vector $v_1$.

Example. Let $v_1,v_2$ be two elements of a vector space $V$. The set of all linear combinations of $v_1,v_2$
$$t_1v_1+t_2v_2: t_1,t_2\in\mathbb{R}\}$$
is a plane through the origin and it is a subspace of $V$, generated by $v_1,v_2$. The plane passing through a point $P\in V$, parallel to $v_1,v_2$ is the set
$$\{P+t_1v_1+t_2v_2: t_1,t_2\in\mathbb{R}\}.$$
However, this is not a subspace unless $P=O$.

A vector space $V$ is a set of objects which can be added and multiplied by numbers, in such a way that the sum of two elements of $V$ is again an element of $V$, the product of an element of $V$ by a number is an element of $V$, and the following properties are satisfied:

VS 1. Given $u,v,w\in V$, we have

$$(u+v)+w=u+(v+w).$$

VS 2. There is an element $O\in V$ such that

$$O+u=u+O=u$$

for all $u\in V$.

VS 3. Given $u\in V$, the element $(-1)u$ is such that

$$u+(-1)u=(-1)u+u=O.$$

$(-1)u$ is simply written as $-u$.

VS 4. For all $u,v\in V$, we have

$$u+v=v+u.$$

VS 5. If $c$ is a number, then $c(u+v)=cu+cv$.

VS 6. If $a,b$ are two numbers, then $(a+b)v=av+bv$.

VS 7. If $a,b$ are two numbers, then $(ab)v=a(bv)$.

VS 8. For any $u\in V$, we have $1u=u$.

The axioms VS 1-VS 4 say that $(V,+)$ is an abelian group. The elements of vector space $V$ are called vectors. One can easily verify that vectors in $\mathbb{R}^n$ satisfy the axioms VS 1-VS 8 and hence $\mathbb{R}^n$ is a vector space.

Example. Let $M(m,n)$ denote the set of all $m\times n$ matrices. Then $M(m,n)$ is a vector space. Using the identification

$$\begin{pmatrix}a_{11} & a_{12} & \cdots & a_{1n}\\a_{21} & a_{22} & \cdots & a_{2n}\\\vdots &\vdots& \ddots&\vdots\\a_{m1} & a_{m2} & \cdots & a_{mn}\end{pmatrix}\longleftrightarrow(a_{11},\cdots,a_{1n};a_{21},\cdots,a_{2n};\cdots;a_{m1},\cdots,a_{mn}),$$

we see that $M(m,n)$ may be identified with $\mathbb{R}^{mn}$ as a vector space.

Example. Let $\mathcal{F}$ be the set of all functions from $\mathbb{R}$ to $\mathbb{R}$. For any $f,g\in\mathcal{F}$, define $f+g$ by

$$(f+g)(x)=f(x)+g(x)$$

for all $x\in\mathbb{R}$. For any $f\in\mathbb{R}$ and for any number $c$, define $cf$ by

$$(cf)(x)=cf(x)$$

for all $x\in\mathbb{R}$. Then $\mathcal{F}$ is a vector space called a function space.

Example. Let $V=\{ae^t+be^{2t}: a,b\in\mathbb{R}\}$. Then $V$ is a vector space. Note that $V$ is the set of all solutions of the second order linear differential equation $\frac{d^2x}{dt^2}-3\frac{dx}{dt}+2x=0$.

Subspaces

A subset $U$ of a vector space $V$ is said to be a subspace if $U$ itself is also a vector space. For $U$ to be a vector space, it suffices to satisfy that

(i) For any $v,w\in U$, $v+w\in U$.

(ii) If $v\in U$ and $c$ is a number, $cv\in U$.

(iii) The identity element $O$ of $V$ is also an element of $U$.

Proposition. A nonempty subset $U$ of a vector space $V$ is a subspace if and only if $av+bw\in U$ for any $v,w\in U$ and numbers $a,b$.

Proof. Exercise

Example. Let $U$ be the set of vectors in $\mathbb{R}^n$ whose last coordinate is $0$. Then $U$ is a subspace of $\mathbb{R}^n$. $U$ may be identified with $\mathbb{R}^{n-1}$.

Example. Let $A$ be a vector in $\mathbb{R}^n$. Let $U$ be the set of all vectors $B$ in $\mathbb{R}^n$ such that $B\cdot A=0$ i.e. $B$ is perpendicular to $A$. Then $U$ is a subspace of $V$.

Example. Let $U$ and $W$ be subspaces of a vector space $V$. Then $U\cap W$ is also a subspace of $V$.

Example. Let $U$ and $W$ be subspaces of a vector space $V$. Define the sum of $U$ and $W$

$$U+W=\{u+w: u\in U\ \rm{and}\ w\in W\}.$$

Then $U+W$ is a subspace.