Consider an $m\times n$ matrix

$$A=\begin{pmatrix}a_{11} & \cdots & a_{1n}\\\vdots & & \vdots\\a_{m1} & \cdots & a_{mn}\end{pmatrix}.$$

The columns of $A$ generate a vector space, which is a subspace of $\mathbb{R}^m$, called the column space of $A$. The dimension of the subspace is called the column rank of $A$. Similarly the rows of $A$ generate a subspace of $\mathbb{R}^n$, called the row space of $A$ and the dimension of this subspace is called the row rank of $A$. It turns out that the column rank and the row rank must be equal. So, we simply call the column rank or the row rank of $A$, the rank of $A$.

There are a couple important theorems regarding the rank of a matrix. They are introduced without proofs.

Theorem. Row and column operations do not change the row rank of a matrix, nor do they change the column rank.

Remark. Row and column operations only change basis of row space or column space.

Theorem. Let $A$ be a matrix of rank $r$. By a succession of row and column operations, the matrix can be transformed to the matrix having components equal to $1$ on the diagonal of the first $r$ rows and columns, and $0$ everywhere else.

$$\begin{pmatrix}1 & 0 & \cdots & 0 & 0 &\cdots &0\\0 & 1 & \cdots & 0 & 0 & \cdots &0\\\vdots & &\ddots &\vdots&\vdots& &\vdots\\0 & 0 &\cdots & 1& 0 &\cdots &0\\0 & 0 &\cdots & 0& 0 &\cdots &0\\\vdots & & &\vdots&\vdots&\ddots&\vdots\\0 & 0 &\cdots & 0& 0 &\cdots &0\end{pmatrix}$$

Example. Find the rank of the matrix $\begin{pmatrix}2 & 1 & 1\\0 & 1 & -1\end{pmatrix}$.

Solution. There are only two rows, so the rank will be at most 2. On the other hand, the column vectors $\begin{pmatrix}2\\0\end{pmatrix}$ and $\begin{pmatrix}1\\1\end{pmatrix}$ are linear independent. Therefore, the rank is 2.

Example. Find the rank of the matrix

$$\begin{pmatrix}1 & 2 & -3\\2 & 1 & 0\\-2 & -1 & 3\\-1 & 4 & -2\end{pmatrix}.$$

Solution. Since there are three columns, the rank will be at most 3. Subtract 2 times column 1 from column 2; add 3 times column 1 to column 3. The resulting matrix is

$$\begin{pmatrix}1 & 0 & 0\\2 & -3 & 6\\-2 & 3 & -3\\-1 & 6 & -5\end{pmatrix}.$$

Add 2 times column 2 to column 3. The resulting matrix is

$$\begin{pmatrix}1 & 0 & 0\\2 & -3 & 0\\-2 & 3 & 3\\-1 & 6 & 7\end{pmatrix}.$$

This matrix is in column echelon form and one can easily see that the first three rwo vectors are linearly independent. Therefore, the rank is 3.