First-Order Linear Differential Equations

The first-order differential equation $\frac{dy}{dx}=f(x,y)$ is called linear if $f(x,y)$ is linear in $y$. Hence, a first-order linear differential equation can be written as \begin{equation}\label{eq:folde}\frac{dy}{dx}+P(x)y=Q(x)\end{equation} If $Q(x)=0$, \eqref{eq:folde} is separable. If $P(x)$ and $Q(x)$ both are constants, \eqref{eq:folde} is separable. Now we assume that \eqref{eq:folde} is not separable. Consider a function $\mu(x)$ which is unbeknown to us at the moment and multiply \eqref{eq:folde} by $\mu(x)$. \begin{equation}\label{eq:folde2}\mu(x)\frac{dy}{dx}+\mu(x)P(x)y=\mu(x)Q(x)\end{equation} On the other hand, we have $$\frac{d\mu y}{dx}=\frac{d\mu }{dx}y+\mu\frac{dy}{dx}$$ So \eqref{eq:folde2} can be writtend as \begin{equation}\label{eq:folde3}\frac{d\mu y}{dx}+y\left[-\frac{d\mu}{dx}+\mu P\right]=\mu Q\end{equation}  If \begin{equation}\label{eq:intfact}-\frac{d\mu}{dx}+\mu P=0\end{equation} then \eqref{eq:folde3} becomes separable and its solution is given by \begin{equation}\label{eq:foldesol}y=\frac{\int\mu Qdx}{\mu}\end{equation} \eqref{eq:intfact} is separable and its solution $\mu(x)$ is given by \begin{equation}\label{eq:intfact2}\mu(x)=e^{\int P(x)dx}\end{equation} The function $\mu(x)$ in \eqref{eq:intfact2} is called the integrating factor.

Example. Solve $\frac{dy}{dx}\sin x=y\cos x+\tan x$

Solution. First write the equation in the standard form \eqref{eq:folde}. $$\frac{dy}{dx}-\cot x y=\sec x$$ Hence $P(x)=-\cot x$ and $Q(x)=\sec x$. $$\int P(x)dx=-\int\cot xdx=-\ln \sin x=\in\csc x$$ and so the integrating factor $\mu(x)$ is given by $$\mu(x)=e^{\ln \csc x}=\csc x$$ and \begin{align*}y(x)&=\frac{\int\mu Qdx}{\mu}\\&=\frac{\csc x\sec xdx}{\csc x}\\&=\sin x\int\frac{dx}{\sin x\cos x}\\&=\sin x[\ln(\tan x)+C]\end{align*}

Example. Solve the initial value problem \begin{align*}ty’+2y&=4t^2\\y(1)&=2\end{align*}

Solution. First write the equation in the standard form \eqref{eq:folde}. $$y’+\frac{2}{t}y=4t$$ and so $P(t)=\frac{2}{t}$ and $Q(t)=4t$. The integrating factor $\mu(t)$ is given by $$\mu(t)=e^{\int\frac{2}{t}dt}=e^{\ln t^2}=t^2$$ \begin{align*}y&=\frac{\int\mu Qdx}{\mu}\\&=\frac{\int 4t^3dt}{t^2}\\&=t^2+\frac{C}{t^2}\end{align*} Since $y(1)=2$, 2=1+C i.e. C=1. Therefore the solution to the initial value problem is $$y(t)=t^2+\frac{1}{t^2}$$

Example. [RL Circuit] For a resistance-inductance circuit, Kirchhoff’s law leads to the first-order linear differential equation $$L\frac{dI(t)}{dt}+RI(t)=V(t)$$ for the current $I(t)$, where $L$ is the inductance and $R$ is the resistance, both constants. $V(t)$ is the time-dependent impressed voltage. The integrating factor $\mu(t)$ is given by $$\mu(t)=e^{\int\frac{R}{L}dt}=e^{\frac{R}{L}t}$$ and hence the current $I(t)$ is found to be \begin{align*}I(t)&=\frac{\int\mu Qdt}{\mu}\\&=\frac{\int e^{\frac{R}{L}t}\frac{V(t)}{L}dt}{e^{\frac{R}{L}t}}\\&=\frac{e^{-\frac{R}{L}t}}{L}\int e^{\frac{R}{L}t}V(t)dt\end{align*} For the special case $V(t)=V_0$, a constant $$I(t)=\frac{V_0}{R}+e^{-\frac{R}{L}t}C$$ If the initial condition is $I(0)=0$, then $C=-\frac{V_0}{R}$ and hence $$I(t)=\frac{V_0}{R}(1-e^{-\frac{R}{L}t})$$

First-Order Differential Equations: Separable Case

What is a differential equation?

A differential equation is an equation that connects the unknown function $y=y(x)$ and its derivatives $$F(x,y,y’,y^{\prime\prime},\cdots)=0$$ where $F$ is a known function. The order of a differential equation is the order of the highest derivative which appears in the equation.

A first-order differential equation is an equation of the form $$F(x,y,y’)=0$$ or $$y’=f(x,y)$$ The simplest case would be $$y’=f(x)$$ in which case the solution is given by $$y=\int f(x)dx$$ Suppose now that $f(x,y)$ is separable, namely $$f(x,y)=g(x)h(y)$$ $\frac{dy}{dx}=g(x)h(y)$ is called a differential equation with variable separable.

Example. Solve $\frac{dy}{dx}=xy$.

Solution. The equation can be written as \begin{equation}\label{eq:sepeq}\frac{dy}{y}=xdx\end{equation} Integrating \eqref{eq:sepeq} i.e. $$\int\frac{dy}{y}=\int xdx$$ we obtain $$\ln y=\frac{x^2}{2}+C’$$ where $C’$ is a constant. Hence the solution is $y$ is given by \begin{align*}y&=e^{\frac{x^2}{2}+C’}\\&=e^{C’}e^{\frac{x^2}{2}}\\&=Ce^{\frac{x^2}{2}}\end{align*} where $C=e^{C’}$.

The solution of a first-order differential equation comes with an undetermined constant. Such a constant can be determined by a so-called initial condition $y(x_0)=y_0$.

Example. Find the solution of $$\frac{x}{1+x^2}dx-\frac{y^2}{1+y^3}dy=0$$ with the initial condition $y(0)=1$.

Solution. \begin{align*}\int\frac{x}{1+x^2}dx-\int\frac{y^2}{1+y^3}dy=0&\Longrightarrow \frac{1}{2}\ln(1+x^2)-\frac{1}{3}\ln(1+y^3)=\frac{1}{6}\ln C\\&\Longrightarrow\frac{(1+x^2)^3}{(1+y
^3)^2}=C\end{align*} where $C$ is a constant. For $x=0$ and $y=1$, $C=\frac{1}{4}$. Therefore the solution is $$(1+y^3)^2-4(1+x^2)^3=0$$

One of the reasons we study differential equations is that many natural phenomena are modeled by differential equations. These models not only tell you what is going on with the natural phenomena but also can predict what is going to happen with the phenomena in the future. We discuss some examples of simple such modelings by differential equations.

Example. A bullet with speed $v_0=400$m/s travels through a wall of thickness $h=20$cm. When the bullet comes out of the wall, its speed is $v_1=100$m/s. If the wall’s resistance is proportional to $v^2$, the square of speed, how long does the bullet travel inside the wall?

Solution. By Newton’s second law of motion the resistance is $$m\frac{dv}{dt}=-kv^2$$ which can be written as $$\frac{dv}{v^2}=-\frac{k}{m}dt$$ Set $k_1=\frac{k}{m}$ and integrate. $$\int\frac{dv}{v^2}=-\int k_1dt$$ and it results in \begin{equation}\label{eq:bullet}\frac{1}{v}=k_1t+C\end{equation} where $C$ is a constant. If $v(0)=v_0$ then $C=\frac{1}{v_0}$. If $v(T)=v_1$ then \begin{equation}\label{eq:bullet2}\frac{1}{v_1}=k_1T+\frac{1}{v_0}\end{equation} Solving \eqref{eq:bullet2} for $T$, $$T=\frac{1}{k_1}\left(\frac{1}{v_1}-\frac{1}{v_0}\right)$$ \eqref{eq:bullet} can be written as $$\frac{1}{v}=\frac{k_1v_0t+1}{v_0}$$ which can  then be written as the differential equation $$\frac{dx}{dt}=\frac{v_0}{k_1v_0t+1}$$ By the integral $$\int dx=\int\frac{v_0}{k_1v_0t+1}dt$$ we find $$x(t)=\frac{1}{k_1}\ln(k_1v_0t+1)+C_1$$ where $C_1$ is a constant. Since $x(0)=0$, $C_1=0$. $x(T)=h$ results in \begin{equation}\label{eq:wall}h=\frac{1}{k_1}\ln(k_1v_0T+1)\end{equation} Solving \eqref{eq:wall} the bullet’s traveling time $T$ within the wall is $$T=\frac{h}{\ln\left(\frac{v_0}{v_1}\right)}\left(\frac{1}{v_1}-\frac{1}{v_0}\right)$$ With $h=20$cm=0.2m, $v_0=400$m/s, and $v_1=100$m/s, we obtain $T=0.00108$ seconds.

Example. Boyle’s gas law is $$\frac{dV}{dP}=-\frac{V}{P}$$ for the volume $V$ of a fixed quantity of gas at pressure $P$ (and constant temperature). The integral $$\int\frac{dV}{V}=-\int\frac{dP}{P}$$ results in $$PV=k$$ where $k$ is a constant.

Example. [Radioactive Decay] The Basic Law of Radioactive Decay: The ratio of the number of atoms disintegrated in unit time to the total number of atoms is a constant that depends only  on the species of atom. This ratio is called the probability of disintegration. Denote by $N(t)$ the quantity of atoms that have not disintegrated by time $t$. Then we have the differential equation $$\frac{dN}{dt}=-\lambda N$$ Here $\lambda$ is the probability disintegration. With the initial condition $N(t_0)=N_0$, the solution is given by $$N(t)=N_0e^{-\lambda t}$$ Radioactive atoms are characterized by their half-life $T$. Haf-life is the time during which the number of atoms $N$ reduces by half the original amount via disintegration. \begin{align*}\frac{N_0}{2}&=N_0e^{-\lambda T}\\T&=\frac{\ln 2}{\lambda}\approx\frac{0.693}{\lambda}\end{align*} There is another important quantity that characterizes radioactive atoms. It is

The mean lifetime $\tau$

Suppose that when an radioactive element was created ($t=0$), there were $N_0$ atoms. During the time interval $[t,t+dt]$ the quantity of atoms that disintegrated is approximately $$-dN=\lambda Ndt$$ All the atoms in this group lived roughly the same lifetime $t$. Among the atoms taken at $t=0$, there are groups of atoms that will have different lifetimes. The mean lifetime $\tau$ is given by $$\tau=\frac{\int_0^\infty t\lambda Ndt}{\int_0^\infty\lambda Ndt}=\frac{1}{\lambda}$$ i.e. the mean lifetime is exactly the inverse of the disintegration probability. $N(t)$ can be also written in terms of $\tau$ as $$N(t)=N_0e^{-\frac{t}{\tau}}$$ Half-life and mean lifetime are related by $$\tau=\frac{T}{\ln 2}\approx 1.443T$$ Radium has a half-life of 1,600 years. Thus \begin{align*}N&=N_0e^{-\frac{\ln 2}{1600}t}\\&=N_0e^{-0.00043t}\end{align*} For $t=200$, $$N(200)=N_0e^{-0.00043\times 200}\approx 0.918N_0$$ This means about 8.2% of the original amount of radium will be disintegrated in 200 years.

Example. [Lanchaster’s Law] Suppose that two armies, Red and Blue are engaging each other in combat. Red is shooting a continuous stream of bullets at Blue. Likewise Blue is shooting a continuous stream of bullets at Red. Assume that the rate of attrition depends only on the number of weapons shooting. Let $A$ represent the number of soldiers in the Red and $B$ represent the number of soldiers in the Blue army. Then we have the coupled system of differential equations \begin{align}\label{eq:lanchaster}\frac{dA}{dt}&=-\beta B\\\label{eq:lanchaster2}\frac{dB}{dt}&=-\alpha A\end{align} The constants $\alpha$ and $\beta$ are called offensive firepower of $A$ and $B$, respectively. Now divide \eqref{eq:lanchaster} by \eqref{eq:lanchaster2}. \begin{align*}\frac{dA}{dB}&=\frac{\beta}{\alpha}\frac{B}{A}\\&=E\frac{B}{A}\ (E=\frac{\beta}{\alpha})\end{align*} By the integral $$\int AdA=E\int BdB$$ we obtain $$A^2=EB^2+C$$ where $C$ is a constant. Suppose that $\alpha=\beta$ i.e. $E=1$ which means the Red and the Blue have the same firepower. Let $A(0)=100$ and $B(0)=50$. Then $C=7500$. If $B=0$, then $A=\sqrt{7500}\approx 86.6$. This means that if all soldiers in the Blue army get killed, the Red army would sustain the casualties of 13 deaths and 1 severely injured.

Trigonometric Substitution

When the integrand contains $\sqrt{a^2+x^2}$, $\sqrt{a^2-x^2}$ or $\sqrt{x^2-a^2}$ and the integral cannot be evaluated by a simple substitution, the following trigonometric substitutions may come in handy:

  1. For $\sqrt{a^2+x^2}$, use $x=a\tan\theta$ with $-\frac{\pi}{2}<\theta<\frac{\pi}{2}$. Accordingly $dx=a\sec^2\theta d\theta$ and $a^2+x^2=a^2\sec^2\theta$.
  2. For $\sqrt{a^2-x^2}$, use $x=a\sin\theta$ with $-\frac{\pi}{2}\leq x\leq\frac{\pi}{2}$. Accordingly $dx=a\cos\theta d\theta$ and $a^2-x^2=a^2\cos^2\theta$. Note: You can also use the substitution $x=a\cos\theta$ with $0\leq\theta\leq\pi$. Accordingly $dx=-a\sin\theta d\theta$ and $a^2-x^2=a^2\sin^2\theta$.
  3. For $\sqrt{x^2-a^2}$, use $x=a\sec\theta$ with $0\leq\theta<\frac{\pi}{2}$ or $\pi\leq\theta<\frac{3\pi}{2}$. Accordingly $dx=a\sec\theta\tan\theta d\theta$ and $x^2-a^2=a^2\tan^2\theta$.

Example. Evaluate $\int\frac{dx}{\sqrt{4+x^2}}$.

Solution. Let $x=2\tan\theta$. Then $dx=2\sec^2\theta d\theta$ and $4+x^2=4\sec^2\theta$. So \begin{align*}\int\frac{dx}{\sqrt{4+x^2}}&=\int\frac{2\sec^2\theta d\theta}{2|\sec\theta|}\\&=\int\sec\theta d\theta\ \left(\sec\theta>0\ \mbox{for}\ -\frac{\pi}{2}<\theta<\frac{\pi}{2}\right)\\&=\ln|\sec\theta+\tan\theta|+C’\end{align*} SInce $\tan\theta=\frac{x}{2}$ one may consider the triangle

Hence $$\int\frac{dx}{\sqrt{4+x^2}}=\ln|\sqrt{4+x^2}+x|+C$$ where $C=C’-\ln 2$.

Example. Evaluate $\int\frac{x^2}{\sqrt{9-x^2}}dx$.

Solution. Let $x=3\sin\theta$. Then $dx=3\cos\theta d\theta$ and $9-x^2=9\cos^2\theta$. \begin{align*}\int\frac{x^2}{\sqrt{9-x^2}}dx&=\int\frac{9\sin^2\theta(3\cos\theta d\theta)}{3|\cos\theta|}\\&=9\int\sin^2\theta d\theta\ \left(\cos\theta>0\ \mbox{for}\ -\frac{\pi}{2}<\theta<\frac{\pi}{2}\right)\\&=9\int\frac{1-\cos 2\theta}{2}d\theta\\&=\frac{9}{2}(\theta-\sin\theta\cos\theta)+C\\&=\frac{9}{2}\left(\sin^{-1}\frac{x}{3}-\frac{x}{3}\frac{\sqrt{9-x^2}}{3}\right)+C\\&=\frac{9}{2}\sin^{-1}\frac{x}{3}-\frac{x}{2}\sqrt{9-x^2}+C\end{align*}

Example. Evaluate $\int\frac{dx}{\sqrt{25x^2-4}}$.

Solution. $\int\frac{dx}{\sqrt{25x^2-4}}=\frac{1}{5}\int\frac{dx}{\sqrt{x^2-\left(\frac{2}{5}\right)^2}}$. Let $x=\frac{2}{5}\sec\theta$. Then $dx=\frac{2}{5}\sec\theta\tan\theta d\theta$ and $x^2-\left(\frac{2}{5}\right)^2=\frac{4}{25}\tan^2\theta$. \begin{align*}\int\frac{dx}{\sqrt{25x^2-4}}&=\frac{1}{5}\int\frac{dx}{\sqrt{x^2-\left(\frac{2}{5}\right)^2}}\\&=\frac{1}{5}\int\frac{\frac{2}{5}\sec\theta\tan\theta d\theta}{5\left(\frac{2}{5}\right)\tan\theta}\ \left(\tan\theta>0\ \mbox{for}\ 0<\theta<\frac{\pi}{2}\right)\\&=\frac{1}{5}\int\sec\theta d\theta\\&=\frac{1}{5}\ln|\sec\theta +\tan\theta|+C’\\&=\frac{1}{5}\ln|5x+\sqrt{25x^2-4}|+C\end{align*} where $C=C’-\frac{\ln 2}{5}$.

Example. Evaluate $\int\sqrt{1+x^2}dx$.

Solution. Let $x=\tan\theta$. Then $dx=\sec^2\theta d\theta$ and $1+x^2=\sec^2\theta$. \begin{align*}\int\sqrt{1+x^2}dx&=\int\sec^3\theta d\theta\\&=\frac{1}{2}\sec\theta\tan\theta+\frac{1}{2}\ln|\sec\theta+\tan\theta|+C\\&=\frac{1}{2}x\sqrt{1+x^2}+\frac{1}{2}\ln|x+\sqrt{1+x^2}|+C\end{align*} The integration $$\int\sec^3\theta d\theta=\frac{1}{2}\sec\theta\tan\theta+\frac{1}{2}\ln|\sec\theta+\tan\theta|+C$$ was discussed here.

Example. Evaluate $\int\frac{\sqrt{1+x^2}}{x}dx$.

Solution. Let $x=\tan\theta$. Then $dx=\sec^2\theta d\theta$ and $1+x^2=\sec^2\theta$. \begin{align*}\int\frac{\sqrt{1+x^2}}{x}dx&=\int\frac{\sec^3\theta}{\tan\theta}d\theta\\&=\int\frac{\sec\theta\sec^2\theta}{\tan\theta}d\theta\\&=\int\frac{\sec\theta(1+\tan^2\theta)}{\tan\theta}d\theta\\&=\int\frac{\sec\theta}{\tan\theta}d\theta+\int\sec\theta\tan\theta d\theta\\&=\int\csc\theta d\theta+\int\sec\theta\tan\theta d\theta\\&=-\ln|\csc\theta+\cot\theta|+\sec\theta+C\\&=-\ln\left|\frac{\sqrt{1+x^2}}{x}+\frac{1}{x}\right|+\sqrt{1+x^2}+C\end{align*}

Example. Find the area enclosed by the ellipse $$\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$$

Solution. The ellipse in the first quadrant is given by the function $y=\frac{b}{a}\sqrt{a^2-x^2}$, $0\leq x\leq a$. So the area is \begin{align*}A&=4\int_0^a\frac{b}{a}\sqrt{a^2-x^2}dx\\&=4\frac{b}{a}\int_0^{\frac{\pi}{2}}a\cos\theta(a\cos\theta d\theta)\ (x=a\sin\theta$)\\&=4ab\int_0^{\frac{\pi}{2}}\cos^2\theta d\theta\\&=4ab\int_0^{\frac{\pi}{2}}\frac{1+\cos 2\theta}{2}d\theta\\&=\pi ab\end{align*}

L’Hôpital’s Rule

If two functions $f(x)$ and $g(x)$ both approach zero as $x\to a$, the function $\frac{f(x)}{g(x)}$ is said to assume the indeterminate form $\frac{0}{0}$ at $x=a$. Regardless of the term, $\frac{f(x)}{g(x)}$ may approach a limit as $x$ approaches $a$. The following theorem named after the French mathematician G. F. A. de L’Hôpital (1661-1704) is useful for the process of determining this limit if it exists.

Theorem (L’Hôpital’s Rule). If the functions $f(x)$ and $g(x)$ are continuous in an interval containing $a$ and if $f'(x)$ and $g'(x)$ exist such that $g'(x)\ne 0$ in this interval (except possibly at $x=a$), then when $\lim_{x\to a}f(x)=\lim_{x\to 0}g(x)=0$ (or equivalently $f(a)=g(a)=0$), we have $$\lim_{x\to a}\frac{f(x)}{g(x)}=\lim_{x\to a}\frac{f'(x)}{g'(x)}$$ provided the limit on the right exists.

Example. Evaluate $\lim_{x\to 0}\frac{\tan x}{x}$.

Solution. The limit is an indeterminate form $\frac{0}{0}$. By applying L’Hôpital’s Rule, we obtain $$\lim_{x\to 0}\frac{\tan x}{x}=\lim_{x\to 0}\frac{\sec^2x}{1}=1$$

The Indeterminate form $\frac{\infty}{\infty}$

If $f(x)\to\infty$ and  $g(x)\to \infty$ as $x\to a$ (or $x\to\pm\infty$), the function $\frac{f(x)}{g(x)}$ is said to assume the indeterminate form $\frac{\infty}{\infty}$ at $x=a$ (or at $x=\pm\infty$). The limit of $\frac{\infty}{\infty}$ as $x\to a$ (or $x\to\pm\infty$) may still be found by L’Hôpital’s Rule if it exists.

Example. $$\lim_{x\to\infty}\frac{x^2}{e^x}=\lim_{x\to\infty}\frac{2x}{e^2}=\lim_{x\to\infty}\frac{2}{e^x}=0$$

Example. Evaluate $\lim_{x\to 0+}\frac{e^{-\frac{1}{x}}}{x}$.

Solution. Applying L’Hôpital’s Rule, we have $$\lim_{x\to 0+}\frac{e^{-\frac{1}{x}}}{x}=\lim_{x\to 0+}\frac{e^{-\frac{1}{x}}}{x^2}$$ While L’Hôpital’s rule can still be applied, we would accomplish nothing by doing so. This example shows that L’Hôpital’s rule may not necessarily leads to a desirable result. For the above example, the limit can be found by $$\lim_{x\to 0+}\frac{e^{-\frac{1}{x}}}{x}=\lim_{x\to 0+}\frac{\frac{1}{x}}{e^{\frac{1}{x}}}=\lim_{z\to\infty}\frac{z}{e^z}=\lim_{z\to\infty}\frac{1}{e^z}=0$$

The indeterminate form $0\cdot\infty$

If $f(x)\to 0$ and $g(x)\to\infty$ as $x\to a$ (or $x\to\pm\infty$), the function $f(x)g(x)$ is said to assume the indeterminate form $0\cdot\infty$ at $x=a$ (or at $x=\pm\infty$). If the limit of $f(x)g(x)$ as $x\to a$ (or $x\to\pm\infty$) exists, it may be found by writing $f(x)g(x)$ as $$\frac{f(x)}{\frac{1}{g(x)}}\ \mbox{or}\ \frac{g(x)}{\frac{1}{f(x)}}$$ and applying L’Hôpital’s rule.

Example.

  1. $\lim_{x\to\infty}xe^{-x}=\lim_{x\to\infty}\frac{x}{e^x}=\lim_{x\to\infty}\frac{1}{e^x}=0$.
  2. $\lim_{x\to 0}\sin 3x\cot 2x=\lim_{x\to 0}\frac{\sin 3x}{\tan 2x}=\lim_{x\to 0}\frac{3\cos 3x}{2\sec^22x}=\frac{3}{2}$.

The Indeterminate Form $\infty-\infty$

If $f(x)\to\infty$ and $g(x)\to\infty$ as $x\to a$ (or $x\to\pm\infty$), the difference $f(x)-g(x)$ is said to assume the indeterminate form $\infty-\infty$  at $x=a$ (or at $x=\pm\infty$). If the limit of $f(x)-g(x)$ as $x\to a$ (or as $x\to\pm\infty$) exists, it may be found by transforming the difference into a fraction by algebraic means and applying L’Hôpital’s rule.

Example. $\lim_{x\to 0}(\csc x-\cot x)=\lim_{x\to 0}\frac{1-\cos x}{\sin x}=\lim_{x\to 0}\frac{\sin x}{\cos x}=0$

Example. Evaluate $\lim_{x\to\infty}(x-\ln x)$.

Solution. While $x-\ln x$ assumes the indeterminate form $\infty-\infty$ at $x=\infty$, there is no algebraic means to transform it to a fraction. An indeterminate of the form $\infty-\infty$ such as the one in consideration may be evaluated by finding the limit of its exponential. Let $y=x-\ln x$. Then $$e^y=e^{x-\ln x}=\frac{e^x}{e^{\ln x}}=\frac{e^x}{x}$$ Hence, $$\lim_{x\to\infty}e^y=\lim_{x\to\infty}\frac{e^x}{x}=\lim_{x\to\infty}\frac{e^x}{1}=\infty$$ Since $y\to\infty$ when $e^y\to\infty$, $$\lim_{x\to\infty}(x-\ln x)=\infty$$

The Indeterminate Forms $0^0$, $\infty^0$, $1^\infty$

If $f(x)\to 0$ and $g(x)\to 0$, or $f(x)\to\infty$ and $g(x)\to 0$, or $f(x)\to 1$ and $g(x)\to\infty$ as $x\to a$ (or $x\to\pm\infty$), $f(x)^{g(x)}$ is said to assume the indeterminate form $0^0$, $\infty^0$, or $1^\infty$, respectively at $x=a$ (or at $x=\pm\infty$). If the limit of $f(x)^{g(x)}$ exists as $x\to a$ (or $x\to\pm\infty$), it may be found by denoting $f(x)^{g(x)}$ by $y$ and investigating the limit approached by the logarithm $$\ln y=g(x)\ln f(x)$$ If $\lim_{x\to a}\ln y=k$, then $\lim_{x\to a}y=e^k$.

Example. Evaluate $\lim_{x\to 0}x^x$.

Solution. $x^x$ assumes the indeterminate form $0^0$ at $x=0$. Let $y=x^x$. Then $\ln y=x\ln x$ and $$\lim_{x\to 0}\ln y=\lim_{x\to 0}\frac{\ln x}{\frac{1}{x}}=\lim_{x\to 0}\frac{\frac{1}{x}}{-\frac{1}{x^2}}=\lim_{x\to 0}(-x)=0$$ Hence $\lim_{x\to 0}y=\lim_{x\to 0}x^x=e^0=1$.

Example. Evaluate $\lim_{x\to 0}(1-\sin x)^{\frac{1}{x}}$.

Solution. The function assumes the indeterminate form $1^\infty$ at $x=0$. Let $y=(1-\sin x)^{\frac{1}{x}}$. Then $\ln y=\frac{\ln(1-\sin x)}{x}$ and $$\lim_{x\to 0}\ln y=\lim_{x\to 0}\frac{\ln(1-\sin x)}{x}=\lim_{x\to 0}\frac{\frac{-\cos x}{1-\sin x}}{1}=-1$$ Hence, $\lim_{x\to 0}(1-\sin x)^{\frac{1}{x}}=e^{-1}=\frac{1}{e}$.

Example. Use L’Hôpital’s rule to show the limit $$\lim_{x\to 0}(1+x)^{\frac{1}{x}}=e$$

Proof. Let $y=(1+x)^{\frac{1}{x}}$. Then $y$ assumes the indeterminate form $1^\infty$ at $x=0$. $\ln y=\frac{\ln(1+x)}{x}$ and $$\lim_{x\to 0}\ln y=\lim_{x\to 0}\frac{\ln(1+x)}{x}=\lim_{x\to 0}\frac{1}{1+x}=1$$ Hence, $$\lim_{x\to 0}(1+x)^{\frac{1}{x}}=e$$

Taylor series

Suppose that $f$ is differentiable infinitely many times on an open interval containing $a$. Then $$\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n=f(a)+\frac{f'(a)}{1!}(x-a)+\frac{f^{\prime\prime}(a)}{2!}(x-a)^2+\frac{f^{\prime\prime\prime}(a)}{3!}(x-a)^3+\cdots$$ is called the Taylor series of $f$ centered at $a$. A Taylor series centered at 0 is called a Maclaurin series.

Example. Find the Maclaurin series of $f(x)=e^x$ and its radius of convergence $R$.

Solution. The Maclaurin series is $\sum_{n=0}^\infty\frac{f^{(n)}(0)}{n!}x^n=\sum_{n=0}^\infty\frac{x^n}{n!}$. By the ratio test $$\lim_{n\to\infty}\left|\frac{a_{n+1}}{a_n}\right|=\lim_{n\to\infty}\frac{|x|}{n+1}=0<1$$ for all $x$ so the series converges for all $x$ i.e. the radius of convergence is $R=\infty$.

The following theorem tells when a function $f(x)$ can be represented by its Taylor series. Recall that $f(x)=T_n(x)+R_n(x)$ where $T_n(x)$ is the $n$-th degree Taylor polynomial of $f$ at $a$ and $R_n(x)=\frac{f^{(n+1)}(\xi)}{(n+1)!}(x-a)^{n+1}$ where $a<\xi<x$ or $x<\xi<a$.

Theorem. $f=\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n$ for $|x-a|<R$ if and only if $\lim_{n\to\infty}R_n(x)=0$ for $|x-a|<R$.

Since $\sum_{n=0}^\infty\frac{x^n}{n!}$ converges for all $x$, \begin{equation}\label{eq:explim}\lim_{n\to\infty}\frac{x^n}{n!}=0\end{equation} Now we show that \begin{equation}\label{eq:expx}e^x=\sum_{n=0}^\infty\frac{x^n}{n!}\end{equation} for all $x$. $R_n(x)=\frac{e^\xi}{(n+1)!}x^{n+1}$ where $0<\xi<x$ or $x<\xi<0$. If $0<\xi<x$ then $0<R_n(x)<\frac{e^x}{(n+1)!}x^{n+1}\to 0$ as $n\to\infty$ by \eqref{eq:explim}. If $x<\xi<0$ then $0\leq |R_n(x)|<\frac{|x|^{n+1}}{(n+1)!}\to 0$ as $n\to\infty$ again by \eqref{eq:explim}. This completes the proof of \eqref{eq:expx}. For $x=1$ we have a definition of the Euler number $e$ in terms of a series as \begin{equation}\label{eq:eulernum}e=\sum_{n=0}^\infty\frac{1}{n!}=1+\frac{1}{1!}+\frac{1}{2!}+\frac{1}{3!}+\cdots\end{equation}

Example. Find the Taylor series of $f(x)=e^x$ at $a=2$.

Solution. By the same manner, one can show that $$e^x=\sum_{n=0}^\infty\frac{e^2}{n!}(x-2)^n$$ for all $x$.

Example. Find the Maclaurin series of $\sin x$ and show that it represents $\sin x$ for all $x$.

Solution. The Maclaurin series is $\sum_{n=0}^\infty\frac{(-1)^nx^{2n+1}}{(2n+1)!}$ and by the ratio test one can show that it converges for all $x$. Since $|f^{(n+1)}(\xi)|\leq 1$, $0\leq |R_n(x)|\leq\frac{|x|^{n+1}}{(n+1)!}\to 0$ as $n\to\infty$. Hence, \begin{equation}\label{eq:sinx}\sin x=\sum_{n=0}^\infty\frac{(-1)^nx^{2n+1}}{(2n+1)!}\end{equation} for all $x$.

Similarly one can also show that \begin{equation}\label{eq:cosx}\cos x=\sum_{n=0}^\infty\frac{(-1)^nx^{2n}}{(2n)!}\end{equation} for all $x$.

Euler’s Formula

If $x$ is replaced by $ix$ in \eqref{eq:expx}, we obtain \begin{align*}e^{ix}&=\sum_{n=0}^\infty\frac{(ix)^n}{n!}\\&=\sum_{n=0}^\infty\frac{x^{2n}}{(2n)!}+i\sum_{n=0}^\infty\frac{x^{2n+1}}{(2n+1)!}\\&=\cos x+i\sin x\end{align*}\begin{equation}\label{eq:eulerformula}e^{ix}=\cos x+i\sin x\end{equation} is called the Euler’s Formula and it represents the point on the unit circle centered at the origin corresponding to the angle $x$. The formula in \eqref{eq:eulerformula} comes in handy in so many places of mathematics from trigonometry, calculus, differential equations to abstract algebra, topology, geometry to name a few. It is also a useful tool in physics on many occasions. For example, one can derive the sine sum and the cosine sum formulas using \eqref{eq:eulerformula}: $e^{i(\theta_1+\theta_2)}=\cos(\theta_1+\theta_2)+i\sin(\theta_1+\theta_2)$.  On the other hand, \begin{align*}e^{i(\theta_1+\theta_2)}&=e^{i\theta_1}e^{i\theta_2}\\&=(\cos\theta_1+i\sin\theta_1)(\cos\theta_2+i\sin\theta_2)\\&=(\cos\theta_1\cos\theta_2-\sin\theta_1\sin\theta_2)+i(\cos\theta_1\sin\theta_2+\sin\theta_1\cos\theta_2)\end{align*} Hence we obtain the sine and the cosine sum formulas \begin{align*}\sin(\theta_1+\theta_2)&=\sin\theta_1\cos\theta_2+\cos\theta_1\sin\theta_2\\\cos(\theta_1+\theta_2)&=\cos\theta_1\cos\theta_2-\sin\theta_1\sin\theta_2\end{align*} When $\theta=\pi$, we obtain the so-called the Euler’s identity \begin{equation}\label{eq:euleridentity}e^{\pi i}+1=0\end{equation} which contains the most fundamental constants of mathematics $\pi$, $e$, $i$, 0, and 1.

The Binomial Series

Let us find the Maclaurin series of $f(x)=(1+x)^k$ where $k$ is any real number. First, we find $$f^{(n)}(x)=k(k-1)\cdots(k-n+1)(1+x)^{k-n}$$ and so $$f^{(n)}(0)=k(k-1)\cdots(k-n+1)$$ Hence the Maclaurin series is given by $$\sum_{n=0}^\infty\frac{k(k-1)\cdots(k-n+1)}{n!}x^n$$ The coefficients $\frac{k(k-1)\cdots(k-n+1)}{n!}$, $n=0,1,2,\cdots$ are denoted by ${}_k\mathrm{C}_n$ or $\begin{pmatrix}k\\n\end{pmatrix}$ and are called binomial coefficients. $$\lim_{n\to\infty}\left|\frac{a_{n+1}}{a_n}\right|=\left|\frac{k-n}{n+1}\right||x|=\lim_{n\to\infty} |x|$$ So by the ratio test if $|x|<1$ then the series converges and if $|x|>1$ then it diverges. Furthermore it can be shown (we omit the proof) that \begin{equation}\label{eq:binomialseries}(1+x)^k=\sum_{n=0}^\infty\frac{k(k-1)\cdots(k-n+1)}{n!}x^n\end{equation} for $|x|<1$. The series in \eqref{eq:binomialseries} is called the binomial series.

Example. FInd the Maclaurin series of $f(x)=\frac{1}{\sqrt{4-x}}$.

Solution. \begin{align*}\frac{1}{\sqrt{4-x}}&=\frac{1}{2}\frac{1}{\sqrt{1-\frac{x}{4}}}\\&=\frac{1}{2}\left(1-\frac{x}{4}\right)^{-\frac{1}{2}}\\&=\frac{1}{2}\sum_{n=0}^\infty\begin{pmatrix}-\frac{1}{2}\\n\end{pmatrix}\left(-\frac{x}{4}\right)^n\\&=\frac{1}{2}\left\{1+\left(-\frac{1}{2}\right)\left(-\frac{x}{4}\right)+\frac{\left(-\frac{1}{2}\right)\left(-\frac{3}{2}\right)}{2!}\left(-\frac{x}{4}\right)^2+\frac{\left(-\frac{1}{2}\right)\left(-\frac{3}{2}\right)\left(-\frac{5}{2}\right)}{3!}\left(-\frac{x}{4}\right)^3+\cdots\\+\frac{\left(-\frac{1}{2}\right)\left(-\frac{3}{2}\right)\left(-\frac{5}{2}\right)\cdots\left(-\frac{2n-1}{2}\right)}{n!}\left(-\frac{x}{4}\right)^n+\cdots\right\}\\&=\frac{1}{2}\left\{1+\frac{x}{8}+\frac{1\cdot 3}{2!8^2}x^2+\frac{1\cdot\cdot 5}{3!8^3}x^3+\cdots+\frac{1\cdot 3\cdot 5\cdots(2n-1)}{n!8^n}x^n+\cdots\right\}\end{align*} This series converges if $\left|-\frac{x}{4}\right|<1$ i.e. $|x|<4$. The radius of convergence is 4.

Working with Taylor Series

Example. Use Taylor series to evaluate $\lim_{x\to 0}\frac{x^2+2\cos x-2}{3x^4}$.

Solution. \begin{align*}\lim_{x\to 0}\frac{x^2+2\cos x-2}{3x^4}&=\lim_{x\to 0}\frac{x^2+2\left(1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\cdots\right)-2}{3x^4}\\&=\lim_{x\to 0}\frac{2\left(\frac{x^4}{4!}-\frac{x^6}{6!}+\cdots\right)}{3x^4}\\&=\frac{2}{4!3}=\frac{1}{36}\end{align*}

Remark. The above limit can be also calculated using the L’Hôpital’s rule as it is an indeterminate form of type $\frac{0}{0}$.

Example. Approximate $\int_0^1 e^{-x^2}dx$ with an error no greater than $5\times 10^{-4}$.

Solution. \begin{align*}\int_0^1 e^{-x^2}dx&=\int_0^1\left(1-x^2+\frac{x^4}{2!}-\frac{x^6}{3!}+\cdots+\frac{(-1)^nx^{2n}}{n!}+\cdots\right)dx\\&=\left[\left(x-\frac{x^3}{3}+\frac{x^5}{2!5}-\frac{x^7}{3!7}+\cdots+\frac{(-1)^nx^{2n+1}}{n!(2n+1)}+\cdots\right)\right]_0^1\\&=1-\frac{1}{3}+\frac{1}{2!5}-\frac{1}{3!7}+\cdots+\frac{(-1)^n}{n!(2n+1)}+\cdots\end{align*} Recall that for an alternating series $\sum_{n=0}^\infty(-1)^n a_n$, $|R_n|\leq a_{n+1}$. For $n=4$, $|R_4|\leq a_5=\frac{1}{5!11}=7.6\times 10^{-3}>5\times 10^{-4}$. For $n=5$, $|R_5|\leq a_6=\frac{1}{6!13}=1.07\times 10^{-4}<5\times 10^{-4}$. Thus the error is less than $5\times 10^{-4}$ if $n\geq 5$. The approximation with $n=5$ is given by \begin{align*}\int_0^1 e^{-x^2}dx&\approx 1-\frac{1}{3}+\frac{1}{2!5}-\frac{1}{3!7}+\frac{1}{5!11}\\&\approx 0.747\end{align*}