Category Archives: Linear Algebra

Determinants as Area and Volume

The Area of a Parallelogram

Let $v=(v_1,v_2)$ and $w=(w_1,w_2)$ be two linearly independent vectors in $\mathbb{R}^2$. Then they span a parallelogram as shown in Figure 1.

Figure 1. Parallelogram spanned by two vector v and w.

The area $A$ of the parallelogram is
\begin{align*}
A&=||v||||w||\sin\theta\\
&=||v\times w||\\
&=\left|\begin{array}{cc}
v_1 & v_2\\
w_1 & w_2
\end{array}\right|.
\end{align*}
In general, the resulting determinant is not necessarily positive. If it is negative, we need to take the absolute value of the determinant for the area.

Exercise. Given two vectors $v=(v_1,v_2,v_3)$ and $w=(w_1,w_2,w_3)$ in $\mathbb{R}^3$, show that $||v||||w||\sin\theta=||v\times w||$ where $\theta$ is the angle between $v$ and $w$.

Hint. Note that $||v||^2||w||^2\sin^2\theta=||v||^2||w||^2-(v\cdot w)^2$.

The Volume of a Parallelepiped

Let $u=(u_1,u_2,u_3)$, $v=(v_1,v_2,v_3)$, $w=(w_1,w_2,w_3)$ be three linearly independent vectors in $\mathbb{R}^3$. Then they span a parallelepiped as shown in Figure 2.

Figure 2. Parallelepiped spanned by vectors u, v and w.

The volume $V$ of the parallelepiped is
\begin{align*}
V&=||u||\cos\theta ||v\times w||\\
&=u\cdot (v\times w)\\
&=\left|\begin{array}{ccc}
u_1 & u_2 & u_3\\
v_1 & v_2 & v_3\\
w_1 & w_2 & w_3
\end{array}\right|.
\end{align*}
In general, the resulting determinant is not necessarily positive. If it is negative, we need to take the absolute value of the determinant for the volume.

Advertisements

Inverse of a Matrix

Let $A$ be an $n\times n$ matrix with $\det A\ne 0$. (A square matrix whose determinant is not equal to $0$ is called non-singular.) Let $X=(x_{ij})$ be an unknown $n\times n$ matrix such that
$AX=I$. Then
$$x_{1j}A^1+\cdots+x_{nj}A^n=E^j.$$
This is a system of linear equations and as we studied here, it can be solved by Cramer’s Rule as
\begin{align*}
x_{ij}&=\frac{\det(A^1,\cdots,E^j,\cdots A^n)}{\det A}\\
&=\frac{1}{\det A}\left|\begin{array}{ccccc}
a_{11} & \cdots & 0 & \cdots & a_{1n}\\
\vdots & & \vdots & & \vdots\\
a_{j1} & \cdots & 1 & \cdots & a_{jn}\\
\vdots & & \vdots & & \vdots\\
a_{n1} & \cdots & 0 & \cdots & a_{nn}
\end{array}\right|\\
&=\frac{1}{\det A}(-1)^{i+j}\det(A_{ji})
\end{align*}
for $i=1,\cdots,n$. If we show that $XA=I$ as well, then $X$ would be the inverse $A^{-1}$ and
$$A^{-1}=\frac{1}{\det A}{}^t((-1)^{i+j}\det(A_{ij})).$$
$\det({}^tA)=\det A\ne 0$, so we can find an $n\times n$ matrix $Y$ such that ${}^tAY=I$. Taking transposes, we obtain ${}^tYA=I$. Now,
$$I={}^tYA={}^tYIA={}^tY(AX)A={}^tYA(XA)=XA.$$

Example. Let $A=\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}$ with $\det A=ad-bc\ne 0$. Then $A^{-1}$ is given by
$$A^{-1}=\frac{1}{ad-bc}\begin{pmatrix}
d & -b\\
-c & a
\end{pmatrix}.$$

Example. Find the inverse of the matrix
$$A=\begin{pmatrix}
3 & 1 & -2\\
-1 & 1 & 2\\
1 & -2 & 1
\end{pmatrix}.$$

Solution. $\det A=16$. We find
\begin{align*}
A_{11}=\begin{pmatrix}
1 & 2\\
-2 & 1
\end{pmatrix},\ A_{12}=\begin{pmatrix}
-1 & 2\\
1 & 1
\end{pmatrix},\ A_{13}=\begin{pmatrix}
-1 & 1\\
1 & -2
\end{pmatrix},\\
A_{21}=\begin{pmatrix}
1 & -2\\
-2 & 1
\end{pmatrix},\ A_{22}=\begin{pmatrix}
3 & -2\\
1 & 1
\end{pmatrix},\ A_{23}=\begin{pmatrix}
3 & 1\\
1 & -2
\end{pmatrix},\\
A_{31}=\begin{pmatrix}
1 & -2\\
1 & 2
\end{pmatrix},\ A_{32}=\begin{pmatrix}
3 & -2\\
-1 & 2
\end{pmatrix},\ A_{33}=\begin{pmatrix}
3 & 1\\
-1 & 1
\end{pmatrix}.
\end{align*}
Hence by the formula we obtain
\begin{align*}
A^{-1}&=\frac{1}{16}{}^t\begin{pmatrix}
\det(A_{11}) & -\det(A_{12}) & \det(A_{13})\\
-\det(A_{21}) & \det(A_{22}) & -\det(A_{23})\\
\det(A_{31}) &-\det(A_{32}) & \det(A_{33})
\end{pmatrix}\\
&=\frac{1}{16}{}^t\begin{pmatrix}
5 & 3 & 1\\
3 & 5 & 7\\
4 & -4 & 4
\end{pmatrix}\\
&=\frac{1}{16}\begin{pmatrix}
5 & 3 & 4\\
3 & 5 &-4\\
1 & 7 & 4
\end{pmatrix}.
\end{align*}

We introduce the following theorem without proof:

Theorem. For any two $n\times n$ matrices $A$, $B$,
$$\det(AB)=\det A\det B.$$

As a special case, we obtain:

Corollary. For an invertible matrix $A$,
$$\det(A^{-1})=\frac{1}{\det A}.$$

Proof. $AA^{-1}=I$, so by the theorem
$$1=\det(AA^{-1})=\det A\det (A^{-1}).$$
Thus proving the formula for the inverse.

Cramer’s Rule

Consider a system of $n$ linear equations in $n$ unknowns
$$x_1A^1+\cdots+x_nA^n=B,$$
where $x_1,\cdots,x_n$ are variables and $A^1,\cdots,A^n,B$ are column vectors of dimension $n$. Suppose that $\det(A^1,\cdots,A^n)\ne 0$. Recall that this condition ensures that the linear system has a unique solution as seen here. Let us consider the determinant of the matrix obtained by replacing $j$-th column $A^j$ by $B$. Then using the properties of determinants studied here
\begin{align*}
\det (A^1,\cdots,B,\cdots,A^n)&=\det (A^1,\cdots,x_1A^1+\cdots+x_nA^n,\cdots,A^n)\\
&=\det (A^1,\cdots,x_1A^1,\cdots,x^nA^n)+\cdots+\det (A^1,\cdots,x_jA^j,\cdots,x_nA^n)\\
&+\cdots+\det (A^1,\cdots,x_nA^n,\cdots,x_nA^n)\\
&=x_1\det (A^1,\cdots,A^1,\cdots,A^n)+\cdots+x_j\det (A^1,\cdots,A^j,\cdots,A^n)\\
&+\cdots+x_n\det (A^1,\cdots,A^n,\cdots,A^n).
\end{align*}
Two column vectors are the same in every term except the $j$-th term, and so every term except the $j$-th term. Hence, we have
$$\det (A^1,\cdots,B,\cdots,A^n)=x_j\det (A^1,\cdots,A^j,\cdots,A^n),$$
i.e.
\begin{align*}
x_j&=\frac{\det (A^1,\cdots,B,\cdots,A^n)}{\det (A^1,\cdots,A^j,\cdots,A^n)}\\
&=\frac{\left|\begin{array}{ccccc}
a_{11} & \cdots & b_1 & \cdots & a_{1n}\\
a_{21} & \cdots & b_2 & \cdots & a_{2n}\\
\vdots & &\vdots& &\vdots\\
a_{n1} & \cdots & b_n & \cdots & a_{nn}
\end{array}\right|}{\left|\begin{array}{ccccc}
a_{11} & \cdots & a_{1j} & \cdots & a_{1n}\\
a_{21} & \cdots & a_{2j} & \cdots & a_{2n}\\
\vdots & &\vdots& &\vdots\\
a_{n1} & \cdots & a_{nj} & \cdots & a_{nn}
\end{array}\right|}
\end{align*}
for $j=1,\cdots,n$. This is called the Cramer’s Rule.

Example. Solve the system of Linear equations:
\begin{align*}
3x+2y+4z&=1,\\
2x-y+z&=0,\\
x+2y+3z&=1.
\end{align*}

Solution. By the Caramer’s Rule, we have
$$x=\frac{\left|\begin{array}{ccc}
1 & 2 & 4\\
0 & -1 & 1\\
1 & 2 & 3
\end{array}\right|}{\left|\begin{array}{ccc}
3 & 2 & 4\\
2 & -1 & 1\\
1 & 2 & 3
\end{array}\right|}=-\frac{1}{5},\ y=\frac{\left|\begin{array}{ccc}
3 & 1 & 4\\
2 & 0 & 1\\
1 & 1 & 3
\end{array}\right|}{\left|\begin{array}{ccc}
3 & 2 & 4\\
2 & -1 & 1\\
1 & 2 & 3
\end{array}\right|}=0,\ z=\frac{\left|\begin{array}{ccc}
3 & 2 & 0\\
2 & -1 & 1\\
1 & 2 & 0
\end{array}\right|}{\left|\begin{array}{ccc}
3 & 2 & 4\\
2 & -1 & 1\\
1 & 2 & 3
\end{array}\right|}=\frac{2}{5}.$$

The Rank of a Matrix 2: The Rank of a Matrix and Subdeterminants

A test for the linear dependence of vectors may be given in terms of determinant.

Theorem. Let $A^1,\cdots,A^n$ be column vectors of dimension $n$. They are linearly dependent if and only if
$$\det(A^1,\cdots,A^n)=0.$$

Corollary. If a system of $n$ linear equations in $n$ unknowns has a matrix of coefficients whose determinants is not 0, then this system has a unique solution.

Proof. A system of $n$ linear equations in $n$ unknowns may be written as
$$x_1A^1+\cdots+x_nA^n=B,$$
where $A^1,\cdots,A^n$ are the column vectors of dimension $n$ of the matrix of coefficients and $B$ is a column vector of dimension $n$. Since $\det(A^1,\cdots,A^n)\ne 0$, $A^1,\cdots,A^n$ are linearly independent by the theorem. So there exists a unique solution $x_1,\cdots,x_n$ of the system.

Since determinants can be used to test linear dependence , they can be also used to determine the rank of a matrix in stead of using row operations as seen here.

Example. Let
$$A=\begin{pmatrix}
3 & 1 & 2 & 5\\
1 & 2 & -1 & 2\\
1 & 1 & 0 & 1
\end{pmatrix}.$$
Since $A$ is a $3\times 4$ matrix, its rank is at most 3. If we can find three linearly independent column vectors, the rank is 3. In fact,
$$\left|\begin{array}{ccc}
1 & 2 & 5\\
2 & -1 & 2\\
1 & 0 & 1
\end{array}\right|=4.$$
So, the rank is exactly 3.

Example. Let
$$B=\begin{pmatrix}
3 & 1 & 2 & 5\\
1 & 2 & -1 & 2\\
4 & 3 & 1 & 7
\end{pmatrix}.$$
Every $3\times 3$ subdeterminant has value 0, so the rank of $B$ is at most 2. The first two rows of $B$. The first two rows are linearly independent since teh determinant
$$\left|\begin{array}{cc}
3 & 1\\
1 & 2
\end{array}\right|$$
is not 0. Hence, the rank is 2.

Determinants II: Determinants of Order $n$

A determinant of order $n$ can be calculated by expanding it in terms of determinants of order $n-1$. Let $A=(a_{ij})$ be an $n\times n$ matrix and let us denote by $A_{ij}$ the $(n-1)\times (n-1)$ matrix obtained by deleting the $i$-th row and the $j$-th column from $A$:

Then $\det A$ is given by the Laplace expansion
\begin{align*}
\det A&=(-1)^{i+1}a_{i1}\det A_{i1}+\cdots+(-1)^{i+n}a_{in}\det A_{in}\\
&=(-1)^{1+j}a_{1j}\det A_{1j}+\cdots+(-1)^{n+j}a_{nj}\det A_{nj}.
\end{align*}

All the properties of the determinants of order 2 we studied here still hold in general for the determinants of order $n$. In particular,

Theorem. Let $A^1,\cdots,A^n$ be column vectors of dimension $n$. They are linearly dependent if and only if
$$\det(A^1,\cdots,A^n)=0.$$

Example. Let us calculate the determinant of the following $3\times 3$ matrix
$$A=\begin{pmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}
\end{pmatrix}.$$
You may use any column or row to calculate $\det A$ using the Laplace expansion. In this example, we use the first row to calculate $\det A$. By the Laplace expansion,
\begin{align*}
\det A&=a_{11}\det A_{11}-a_{12}\det A_{12}+a_{13}\det A_{13}\\
&=a_{11}\left|\begin{array}{cc}
a_{22} & a_{23}\\
a_{32} & a_{33}
\end{array}\right|-a_{12}\left|\begin{array}{cc}
a_{21} & a_{23}\\
a_{31} & a_{33}
\end{array}\right|+a_{13}\left|\begin{array}{cc}
a_{21} & a_{22}\\
a_{31} & a_{32}
\end{array}\right|.
\end{align*}
Replay this with
$$A=\begin{pmatrix}
2 & 1 & 0\\
1 & 1 & 4\\
-3 & 2 & 5
\end{pmatrix}.$$
Since the first row or the third column contains 0, you may want to use the first row or the third column to do the Laplace expansion.

For $3\times 3$ matrices, there is a quicker way to calculate the determinant as shown in the following figure. You multiply three entries along each indicated arrow. When you multiply three entries along each red arrow, you also multiply by $-1$. This is called the Rule of Sarrus named after a French mathematician Pierre Frédéric Sarrus. Please be warned that the rule of Sarrus works only for $3\times 3$ matrices.

The Rule of Sarrus

Example. [Cross Product] Let $v=v_1E_1+v_2E_2+v_3E_3$ and $w=w_1E_1+w_2E_2+w_3E_3$ be two vectors in Euclidean 3-space $\mathbb{R}^3$. The cross product is defined by
$$v\times w=\left|\begin{array}{ccc}
E_1 & E_2 & E_3\\
v_1 & v_2 & v_3\\
w_1 & w_2 & w_3
\end{array}\right|.$$
Note that the cross product is perpendicular to both $v$ and $w$.

Clearly, if there are many 0 entries in a given determinant, it would be easier to calculate the determinant since you will have a lesser than usual number of terms that you actually have to calculate in the Laplace expansion. For any given determinant, we can indeed make it happen. Recall the theorem we studied here:

Theorem. If one adds a scalar multiple of one column (row) to another column (row), then the value of the determinant does not change.

Using the particular column (row) operation in the Theorem, we can turn a given determinant into one with more 0 entries.

Example. Find
$$\left|\begin{array}{cccc}
1 & 3 & 1 & 1\\
2 & 1 & 5 & 2\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|.$$

Solution. By the above Theorem,
\begin{align*}
\left|\begin{array}{cccc}
1 & 3 & 1 & 1\\
2 & 1 & 5 & 2\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
2 & 1 & 5 & 2\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|\ (\mbox{add $-1$ times row 3 to row 1})\\
&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
0 & 3 & 1 & -4\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|\ (\mbox{add $-2$ times row 3 to row 2})\\
&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
0 & 3 & 1 & -4\\
1 & -1 & 2 & 3\\
0 & 5 & -11 & -5
\end{array}\right|\ (\mbox{add $-4$ times row 3 to row 4})\\
&=\left|\begin{array}{ccc}
4 & -1 & -2\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|\ (\mbox{Laplace expansion along column 1})
\end{align*}
You may compute the resulting determinant of order 3 using the rule of Sarrus or you may further simpify it. For instance, you may do:
\begin{align*}
\left|\begin{array}{ccc}
4 & -1 & -2\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|&=\left|\begin{array}{ccc}
7 & 0 & -6\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|\ (\mbox{add row 2 to row 1})\\
&=\left|\begin{array}{ccc}
7 & 0 & -6\\
3 & 1 & -4\\
38 & 0 & -49
\end{array}\right|\ (\mbox{add 11 times row 2 to row 3})\\
&=\left|\begin{array}{cc}
7 & -6\\
38 & -49
\end{array}\right|\ (\mbox{Laplace expansion along column 2})\\
&=-115.
\end{align*}