Bessel’s Inequality

Bessel’s inequality is important in studying Fourier series.

Theorem. If $f$ is $2\pi$-periodic and Riemann integrable on $[-\pi,\pi]$ and if the Fourier coefficients $c_n$ are defined by
$$c_n=\frac{1}{2\pi}\int_{-\pi}^{\pi}f(\theta)e^{-in\theta}d\theta,$$
then
\begin{equation}
\label{eq:besselinequality}
\sum_{n=-\infty}^\infty|c_n|^2\leq\frac{1}{2\pi}\int_{-\pi}^\pi|f(\theta)|^2d\theta.
\end{equation}

Proof.
\begin{align*}
0&\leq|f(\theta)-\sum_{-N}^Nc_ne^{in\theta}|^2\\
&=f(\theta)^2-\sum_{-N}^Nf(\theta)[c_ne^{in\theta}+\overline{c_n}e^{-in\theta}]+\sum_{m,n=-N}^Nc_m\overline{c_n}e^{i(m-n)\theta}
\end{align*}
By integrating,
\begin{align*}
\frac{1}{2\pi}\int_{-\pi}^\pi|f(\theta)-\sum_{-N}^Nc_ne^{in\theta}|^2d\theta&=\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)^2d\theta-\sum_{-N}^N\left[c_n\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)e^{in\theta}d\theta\right.\\
\left.+\overline{c_n}\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)e^{-in\theta}d\theta\right]+&\sum_{m,n=-N}^Nc_m\overline{c_n}\frac{1}{2\pi}\int_{-\pi}^\pi e^{i(m-n)\theta}d\theta\\
&=\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)^2d\theta-\sum_{-N}^N|c_n|^2.
\end{align*}
Hence, for each $N=1,2,\cdots$,
$$\sum_{-N}^N|c_n|^2\leq\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)^2d\theta.$$
Taking the limit $N\to\infty$, we obtain
$$\sum_{-\infty}^\infty|c_n|^2\leq\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)^2d\theta.$$

Note that $|a_0|^2=4|c_0|^2$, $|a_n|^2+|b_n|^2=2(|c_n|+|c_{-n}|^2)$, $n\geq 1$. So, in terms of the real coefficients, Bessel’s inequality can be written as
\begin{equation}
\label{eq:besselinequality2}
\frac{1}{4}|a_0|^2+\frac{1}{2}\sum_1^\infty(|a_n|^2+|b_n|^2)\leq\frac{1}{2\pi}\int_{-\pi}^\pi f(\theta)^2d\theta.
\end{equation}
Bessel’s inequality implies that $\sum|a_n|^2$, $\sum|b_n|^2$, $\sum|c_n|^2$ are convergent and hence the series of Fourier coefficients $\sum a_n$, $\sum b_n$, $\sum c_n$ are convergent. As we studied in undergraduate calculus the following corollary holds then.

Corollary. The Fourier coefficients $a_n$, $b_n$, $c_n$ tend to zero as $n\to\infty$ (and also as $n\to -\infty$ for $c_{-n}$).

Spectrum

Let us recall the Hooke’s law
\begin{equation}
\label{eq:hooke}
F=-kx.
\end{equation}
Newton’s second law of motion is
\begin{equation}
\label{eq:newton}
F=ma=m\ddot{x},
\end{equation}
where $\ddot{x}=\frac{d^2 x}{dt^2}$. The equations \eqref{eq:hooke} and \eqref{eq:newton} result the equation of a simple harmonic oscillator
\begin{equation}
\label{eq:ho}
m\ddot{x}+kx=0.
\end{equation}
Integrating \eqref{eq:ho} with respect to $x$, we have
$$\int(m\ddot{x}dx+kxdx)=E_0,$$
where $E_0$ is a constant. $d\dot{x}=\ddot{x}dt$ and $\dot{x}d\dot{x}=\dot{x}\ddot{x}dt=\ddot{x}dx$. So,
\begin{align*}
\int(m\ddot{x}dx+kxdx)&=\int(m\dot{x}d\dot{x}+kxdx)\\
&=\frac{1}{2}m\ddot{x}+\frac{1}{2}kx^2.
\end{align*}
Hence, we obtain the conservation law of energy
\begin{equation}
\label{eq:energy}
\frac{1}{2}m\ddot{x}+\frac{1}{2}kx^2=E_0.
\end{equation}
The general solution of \eqref{eq:ho} is
\begin{equation}
\label{eq:hosol}
\begin{aligned}
x(t)&=a\cos\omega t+b\sin\omega t\\
&=\sqrt{a^2+b^2}\sin(\omega t+\theta),
\end{aligned}
\end{equation}
where $a$ and $b$ are constants, $\omega=\sqrt{\frac{k}{m}}$ and $\theta=\tan^{-1}\left(\frac{a}{b}\right)$. From \eqref{eq:energy} and \eqref{eq:hosol}, the total energy $E_0$ is computed to be
$$E_0=\frac{1}{2}m\omega^2(a^2+b^2).$$
This tells us that the total energy of a simple harmonic oscillator is proportional to $a^2+b^2$, the squared amplitude. As seen here, the sawtooth function $f(x)$ is represented as the Fourier series
\begin{align*}
f(x)&=-\frac{2L}{\pi}\sum_{n=1}^\infty\frac{(-1)^n}{n}\sin\left(\frac{n\pi x}{L}\right)\\
&=\frac{2L}{\pi}\left\{\sin\left(\frac{\pi x}{L}\right)-\frac{1}{2}\sin\left(\frac{2\pi x}{L}\right)+\frac{1}{3}\sin\left(\frac{3\pi x}{L}\right)-\cdots\right\}.
\end{align*}
The amplitude $c_n=\frac{2L}{n\pi}$, $n=1,2,3,\cdots$ coincides with twice the angular frequency. $\{c_n\}$ is called the frequency spectrum or the amplitude spectrum.

The First and Second Derivative Tests

The First Derivative Test

The derivative $f'(x)$ can tell us a lot about the function $y=f(x)$. It can tell us where critical points are i.e. points at which $f'(x)=0$ and the critical points are likely places at which $y=f(x)$ assumes a local maximum or a local minimum values. By further examining the properties of $f'(x)$ we can also determine at which critical point, $f(x)$ assumes a local maximum, or a local minimum, or neither. But first we see that $f'(x)$ can tell us where $y=f(x)$ is increasing or decreasing.

Theorem. Increasing/Decreasing Test

  1. If $f'(x)>0$ on an open interval, $f$ is increasing on that interval.
  2. If $f'(x)<0$ on an open interval, $f$ is decreasing on that interval.

Example. Find where $f(x)=3x^4-4x^3-12x^2+5$ is increasing and where it is decreasing.

Solution.
\begin{align*}
f'(x)&=12x^3-12x^2-24x\\
&=12x(x^2-x-2)\\
&=12x(x-2)(x+1).
\end{align*}
The critical points are $x=-1,0,2$. Using, for instance, the test point method (which is the easiest method of solving an inequality), we obtain the following table.
$$
\begin{array}{|c|c|c|c|c|c|c|c|}
\hline
x & x<-1 & -1 & -1<x<0 & 0 & 0<x<2 & 2 & x>2\\
\hline
f'(x) & – & 0 & + & 0 & – & 0 & +\\
\hline
f(x) & \searrow & f(-1) & \nearrow & f(0) & \searrow & f(2) &\nearrow\\
\hline
\end{array}
$$
So we find that $f$ is increasing on $(-1,0)\cup(2,\infty)$ and $f$ is decreasing on $(-\infty,-1)\cup(0,2)$.

Now, local maximum values and local minimum values can be identified by observing the change of sign of $f'(x)$ at each critical point.

Theorem. [The First Derivative Test] Suppose that $c$ is a critical point of a differentiable function $f(x)$.

  1. If the sign of $f'(x)$ changes from $+$ to $-$ at $c$, $f(c)$ is a local maximum.
  2. If the sign of $f'(x)$ changes from $-$ to $+$ at $c$, $f(c)$ is a local minimum.
  3. If the sign $f'(x)$ does not change at $c$, $f$ has neither a local maximum nor a local minimum at $c$.

Example. In the previous example, the sign of $f'(x)$ changes from $+$ to $-$ at $0$, so $f(0)=5$ is a local maximum. The sign of $f'(x)$ changes from $-$ to $+$ at $-1$ and at $2$, so $f(-1)=0$ and $f(2)=-27$ are local minimum values.

The following figure confirms our findings from the above two examples.

The graph of f(x)=3x^4-4x^3-12x^2+5

The graph of f(x)=3x^4-4x^3-12x^2+5

The Second Derivative Test

The second order derivative $f^{\prime\prime}(x)$ can provide us an additional piece of information on $y=f(x)$, namely the concavity of the graph of $y=f(x)$.

Definition. If the graph of $f$ lies above all of its tangents on an open interval $I$, it is called concave upward on $I$. If the graph of $f$ lies below all of its tangents on $I$, it is called concave downward on $I$.

From here on, $\smile$ denotes “concave up” and $\frown$ denotes “concave down”.

Definition. A point $(d,f(d))$ on the graph of $y=f(x)$ is called a point of inflection if the concavity of the graph of $f$ changes from $\smile$ to $\frown$ or from $\frown$ to $\smile$ at $(d,f(d))$. The candidates for the points of inflection may be found by solving the equation $f^{\prime\prime}(x)=0$ as shown in the example below.

Theorem. [Concavity Test]

  1. If $f^{\prime\prime}(x)>0$ for all x in an open interval $I$, the graph of $f$ is concave up on $I$.
  2. If $f^{\prime\prime}(x)<0$ for all x in an open interval $I$, the graph of $f$ is concave down on $I$.

Theorem. [The Second Derivative Test] Suppose that $f'(c)=0$ i.e. $c$ is a critical point of $f$. Suppose that $f^{\prime\prime}$ is continuous near $c$.

  1. If $f^{\prime\prime}(c)>0$ then $f(c)$ is a local minimum.
  2. If $f^{\prime\prime}(c)<0$ then $f(c)$ is a local maximum.

Example. Let $f(x)=-x^4+2x^2+2$.

  1. Find and identify all local maximum and local minimum values of $f(x)$ using the Second Derivative Test.
  2. Find the intervals on which the graph of $f(x)$ is concave up or concave down. Find all points of inflection.

Solution. 1. First we find the critical points of $f(x)$ by solving the equation $f'(x)=0$:
$$f'(x)=-4x^3+4x=-4x(x^2-1)=-4x(x+1)(x-1)=0.$$ So $x=-1,0,1$ are critical points of $f(x)$ Next, $f^{\prime\prime}(x)=-12x^2+4$. Since $f^{\prime\prime}(0)=4>0$ and $f^{\prime\prime}(-1)=f^{\prime\prime}(1)=-8<0$, by the Second Derivative Test, $f(0)=2$ is a local minimum value and $f(-1)=f(1)=3$ is a local maximum value.

2. First we need to solve the equation $f”(x)=0$:
$$f^{\prime\prime}(x)=-12x^2+4=-12\left(x^2-\frac{1}{3}\right)=-12\left(x+\frac{1}{\sqrt{3}}\right)\left(x-\frac{1}{\sqrt{3}}\right)=0.$$ So $f^{\prime\prime}(x)=0$ at $x=\pm\displaystyle\frac{1}{\sqrt{3}}$. By using the test-point method we find the following table:
$$
\begin{array}{|c||c|c|c|c|c|}
\hline
x & x<-\frac{1}{\sqrt{3}} & -\frac{1}{\sqrt{3}} & -\frac{1}{\sqrt{3}}<x<\frac{1}{\sqrt{3}} & \frac{1}{\sqrt{3}} & x>\frac{1}{\sqrt{3}}\\
\hline
f^{\prime\prime}(x) & – & 0 & + & 0 & -\\
\hline
f(x) & \frown & f\left(-\frac{1}{\sqrt{3}}\right)=\frac{23}{9} & \smile & f\left(\frac{1}{\sqrt{3}}\right)=\frac{23}{9} & \frown\\
\hline
\end{array}
$$
The graph of $f(x)$ is concave down on the intervals $\left(-\infty,-\frac{1}{\sqrt{3}}\right)\cup\left(\frac{1}{\sqrt{3}},\infty\right)$ and is concave up on the interval $\left(-\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}}\right)$. The points of inflection are $\left(-\frac{1}{\sqrt{3}},\frac{23}{9}\right)$ and $\left(\frac{1}{\sqrt{3}},\frac{23}{9}\right)$.

The following figure confirms our findings from the above example.

The graph of f(x)=-x^4+2x^2+2 with points of inflection (in blue)

The graph of f(x)=-x^4+2x^2+2 with points of inflection (in blue)

The Substitution Rule

If given integration takes the form $\int f(g(x))g'(x)dx$ then it can be converted to a simpler integration that we may be able to evaluate by the substitution $u=g(x)$. In fact, the integration is given in terms of the new variable $u$ as
$$\int f(g(x))g'(x)dx=\int f(u)du.$$

Example. Evaluate $\int x\sqrt{1+x^2}dx$.

Solution. Let $u=1+x^2$. Then $du=2xdx$. So,
\begin{align*}
\int x\sqrt{1+x^2}dx&=\frac{1}{2}\int\sqrt{u}du\\
&=\frac{1}{3}u^{\frac{3}{2}}+C\\
&=\frac{1}{3}(1+x^2)^{\frac{3}{2}}+C,
\end{align*}
where $C$ is an arbitrary constant.

Example. Evaluate $\int\cos (7\theta+5)d\theta$.

Solution. Let $u=7\theta+5$. Then $du=7d\theta$. So,
\begin{align*}
\int\cos (7\theta+5)d\theta&=\frac{1}{7}\int\cos udu\\
&=\frac{1}{7}\sin u+C\\
&=\frac{1}{7}\sin(7\theta+5)+C,
\end{align*}
where $C$ is an arbitrary constant.

Example. Evaluate $\int x^2\sin(x^3)dx$.

Solution. Let $u=x^3$. Then $du=3x^2dx$. So,
\begin{align*}
\int x^2\sin(x^3)dx&=\frac{1}{3}\int \sin udu\\
&=-\frac{1}{3}\cos u+C\\
&=-\frac{1}{3}\cos(x^3)+C,
\end{align*}
where $C$ is an arbitrary constant.

Example. Integrals of $\sin^2x$ and $\cos^2x$.

  1. \begin{align*}\int\sin^2xdx&=\int\frac{1-\cos 2x}{2}dx\\&=\frac{x}{2}-\frac{\sin 2x}{4}+C\end{align*}
  2. \begin{align*}\int\cos^2xdx&=\int\frac{1+\cos 2x}{2}dx\\&=\frac{x}{2}+\frac{\sin 2x}{4}+C\end{align*}

How do we evaluate a definite integral of the form $\int_a^b f(g(x))g'(x)dx$? The following example shows you how.

Example. Evaluate $\int_{-1}^1 3x^2\sqrt{x^3+1}dx$.

Solution. First let us calculate the indefinite integral $\int 3x^2\sqrt{x^3+1}dx$. Let $u=x^3+1$. Then $du=3x^2dx$. So,
\begin{align*}
\int 3x^2\sqrt{x^3+1}dx&=\int\sqrt{u}du\\
&=\frac{2}{3}u^{\frac{3}{2}}+C\\
&=2(x^3+1)^{\frac{3}{2}}+C,
\end{align*}
where $C$ is an arbitrary constant. Now by Fundamental Theorem of Calculus,
\begin{align*}
\int_{-1}^1 3x^2\sqrt{x^3+1}dx&=\frac{2}{3}[(x^3+1)^{\frac{3}{2}}]_{-1}^1\\
&=\frac{4\sqrt{2}}{3}.
\end{align*}

But there is a better way to do this as shown in the following theorem. Its proof is straightforward.

Theorem. If $u’$ is continuous on $[a,b]$ and $f$ is continuous on the range of $u$, then
$$\int_a^bf(u(x))u'(x)dx=\int_{u(a)}^{u(b)}f(u)du.$$

Example. Let us replay the previous example using this theorem. Again let $u=x^3+1$. Then $du=3x^2dx$ and $u(-1)=0$, $u(1)=2$. Now, by the above theorem,
\begin{align*}
\int_{-1}^1 3x^2\sqrt{x^3+1}dx&=\int_0^2\sqrt{u}du\\
&=\frac{2}{3}[u^{\frac{3}{2}}]_0^2\\
&=\frac{4\sqrt{2}}{3}.
\end{align*}
I believe you will find this more simple than previous method.

Example. Evaluate $\int_{\frac{\pi}{4}}^{\frac{\pi}{2}}\cot\theta\csc^2\theta d\theta$.

Solution. Let $u=\cot\theta$. Then $u\left(\frac{\pi}{4}\right)=1$, $u\left(\frac{\pi}{2}\right)=0$, and $du=-\csc^2\theta d\theta$. Hence, \begin{align*}\int_{\frac{\pi}{4}}^{\frac{\pi}{2}}\cot\theta\csc^2\theta d\theta&=-\int_1^0 udu\\&=\frac{1}{2}\end{align*}

I will finish this lecture with the following nice properties.

Theorem. Let $f$ be a continuous function on $[-a,a]$.

(a) If $f$ is an even function, then $\int_{-a}^a f(x)dx=2\int_0^a f(x)dx$.

(b) If $f$ is an odd function, then $\int_{-a}^a f(x)dx=0$.

This can be easily understood from pictures using the symmetries of even and odd functions. But the theorem can be proved using substitution. I will leave it to you.

Example.  \begin{align*}\int_{-2}^2(x^4-4x^2+6)dx&=2\int_0^2(x^4-4x^2+6)dx\\&=2\left[\frac{x^5}{5}-\frac{4}{3}x^3+6x\right]_0^2\\&=\frac{232}{15}\end{align*}

The graph of y=x^4-4x^2+6 on [-2,2].

Mean Value Theorem

The following theorem is something one can easily picture intuitively.

Theorem. [Rolle’s Theorem]
Let $f$ be continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b)$. If $f(a)=f(b)$, then there exists a number $c$ in $(a,b)$ such that $f'(c)=0$.

Example. Show that the equation $x^3+x-1=0$ has exactly only one real root.

Solution. Let $f(x)=x^3+x-1$. Note that $f(0)=-1$ and $f(1)=1$. So by the Intermediate Value Theorem, we see that there exists at least a root of the equation $x^3+x-1=0$ in the interval $(0,1)$. Now suppose that there are two different roots $a$ and $b$ of the equation $x^3+x-1=0$. Without loss of generality, we may assume that $a<b$. Then $f(x)$ is continuous on $[a,b]$ and differentiable on $(a,b)$. By Rolle’s Theorem then, there exist a number $c$ in $(a,b)$ such that $f'(c)=0$. However, $f'(x)=3x^2+1\geq 1$ for all real number $x$. This is a contradiction. Therefore, there should be only one root of the equation.

The graph of f(x)=x^3+x-1

The graph of f(x)=x^3+x-1

Let $f$ be continuous on $[a,b]$ and differentiable on $(a,b)$. Define $g(x)$ to be the distance between $f(x)$ and the line segment from $(a,f(a))$ to $(b,f(b))$, i.e.
$$g(x)=f(x)-\frac{f(b)-f(a)}{b-a}(x-a)-f(a).$$ Then $g(x)$ is continuous on $[a,b]$ and differentiable on $(a,b)$. Since $g(a)=g(b)=0$, by Rolle’s theorem there exists a number $c$ in $(a,b)$ such that $g'(c)=f'(c)-\frac{f(b)-f(a)}{b-a}=0$. Therefore, we proved the following theorem.

Mean Value Theorem

Mean Value Theorem

Theorem. [Mean Value Theorem]
Let $f$ be continuous on $[a,b]$ and differentiable on $(a,b)$. Then there exists a number $c$ in $(a,b)$ such that
$$f'(c)=\frac{f(b)-f(a)}{b-a}.$$

Remark. Why the name Mean Value Theorem? The average $\mathrm{av}(f)$ of a continuous function $f$ on a closed interval $[a,b]$ can be defined by $$\mathrm{av}(f)=\frac{1}{b-a}\int_a^b f(x)dx$$ See here for details. If $f'(x)$ is continuous on $[a,b]$, its average on $[a,b]$ is given by $$\frac{1}{b-a}\int_a^bf'(x)dx=\frac{f(b)-f(a)}{b-a}$$ That is, Mean Value Theorem states that one of the values of $f'(x)$ on $(a,b)$ becomes the average of $f'(x)$ on $[a,b]$.

The following examples are applications of the Mean Value Theorem.

Example. Suppose that $f(0)=-3$ and $f'(x)\leq 5$ for all values of $x$. How large can $f(2)$ possibly be?

Solution. By the Mean Value Theorem, there exists a number $c$ in $(0,2)$ such that
$$f'(c)=\frac{f(2)-f(0)}{2-0}=\frac{f(2)+3}{2}.$$
Since $f'(c)\leq 5$,
\begin{align*}
f(2)&=2f'(c)-3\\
&\leq 2\cdot 5-3=7.
\end{align*}
Hence, $7$ is the largest possible value of $f(2)$.

Example. A trucker handed in a ticket at a toll booth showing in 2 hours she had covered 159 mi on a toll road with speed limit of 65 mph. The trucker was cited for speeding. Why?

Solution. The average speed was $\frac{159}{2}=79.5$ mph. By the Mean Value Theorem the trucker was driving at the speed 79.5 mph at some point .

Using Mean Value Theorem, one can prove the following theorem. The proof is straightforward and is left for readers.

Theorem. If $f'(x)=0$ for all $x$ in the open interval $(a,b)$, then $f$ is constant on $(a,b)$.