Mathphys Archive@Blogger and Mathphys Archive@Substack

This wordpress site is currently running on an outdated machine along with outdated OS and software. In order for this site to be up-to-date so it can work properly and efficiently, I need to build a new server on a new machine. Unfortunately, due to my on-going battle with cancer, I don’t have resources, time and energy to do that. This made me to decide to cease the operation of this site. No, I am not shutting down this site. This site will remain online but it will no longer be updated with new entries (notes). My writing activity will continue elsewhere though. I opened a new blog site at Blogger. Like my current wordpress site, the new site will have the collection of my online lecture notes on mathematics, physics, and related areas (theoretical computer science, mathematical biology, and mathematical finance) but with a new style. The lecture notes will be focused on ideas, computations, examples, and applications rather than detailed and complex proofs and technicalities. Only short and easy proofs will be included in each lecture. For long, tedious or difficult proofs, sketches of proofs that convey core ideas will be employed. In addition, lecture slides (which summarize the lecture notes) and my lecture videos (each of which has no more than 30 minutes of play time) will be hosted at my Substack site as well as at my YouTube Channel.

My lecture notes at Blogger and lecture videos at Substack and at YouTube will all be available to everybody for free. When ready, I will also have paid subscription option at my Substack site. My paid subscribers will have additional access to exercise solutions, more detailed explanations and discussions on proofs, technicalities and what not, questions and answers, and consultations (guidance) about studying math or physics upon request.

Please join me at my Blogger site, Substack site and YouTube Channel, and if you like what I write and lecture there and think they are helpful for you, also consider being my paid subscriber at my Substack site when it is available.

Note: I am currently away from home undergoing cancer treatments. Unfortunately, I am not properly equipped to make lecture videos for the moment. I will have to wait until I am back home to start making lecture videos.

The Central Limit Theorem

Theorem. (The Central Limit Theorem)

Let $X_1,X_2,\cdots$ be a sequence of independent and identically distributed random variables each having mean $\mu$ and variance $\sigma^2$. Then the distribution of
$$\frac{X_1+\cdots+X_n-n\mu}{\sigma\sqrt{n}}$$
tend to the standard normal as $n\to\infty$. That is, for $-\infty<a<\infty$,
$$P\left\{\frac{X_1+X_2+\cdots+X_n-n\mu}{\sigma\sqrt{n}}\leq a\right\}\to\frac{1}{\sqrt{2\pi}}\int_{-\infty}^a e^{-\frac{x^2}{2}}dx$$
as $n\to\infty$.

The central limit theorem is one of the most remarkable results in probability theory.

We introduce the following lemma without proof, which is crucial in proving the central limit theorem.

Lemma. Let $Z_1, Z_2,\cdots$ be a sequence of random variables having distribution functions $F_{Z_n}$ and moment generating functions $M_{Z_n}$, $n\geq 1$; and let $Z$ be a random variable having distribution function $F_Z$ and moment generating function $M_Z$. If $M_{Z_n}(t)\to M_Z(t)$ for all $t$, then $F_{Z_n}(t)\to F_Z(t)$ for all $t$ at which $F_Z(t)$ is continuous.

Example. If $Z$ is a unit normal random variable (i.e. $\mu=0$ and $\sigma^2=1$), then $M_Z(t)=e^{\frac{t^2}{2}}$. It follows from the lemma that if $M_{Z_n}(t)\to e^{\frac{t^2}{2}}$ as $n\to\infty$, then $F_{Z_n}(t)\to\Phi(t)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^t e^{\frac{-y^2}{2}}dy$.

We now prove the central limit theorem.

Proof. Let us assume at first that $\mu=0$ and $\sigma^2=1$. Also assume that the moment generating function of the $X_i$, $M(t)$ exists and is finite. (Since the random variables $X_1,X_2,\cdots$ are identically distributed, they have the same distributions and hence their moment generating functions are the same.) Let
$$Z_n=\frac{X_1+\cdots+X_n}{\sqrt{n}}=\sum_{i=1}^n\frac{X_i}{\sqrt{n}},\ n=1,2,\cdots$$
Then the moment generating function $M_{Z_n}(t)$ is
\begin{align*} M_{Z_n}(t)&=E\left[\exp\left\{\sum_{i=1}^n\frac{tX_i}{\sqrt{n}}\right\}\right]\\ &=E\left[\prod_{i=1}^n\exp\left\{\frac{tX_i}{\sqrt{n}}\right\}\right]\\ &=\prod_{i=1}^nE\left[\exp\left\{\frac{tX_i}{\sqrt{n}}\right\}\right]\\ (\mbox{the $X_i$’s are independent}) \end{align*}
$E\left[\exp\left\{\frac{tX_i}{\sqrt{n}}\right\}\right]$ is the moment generating function of $\frac{X_i}{\sqrt{n}}$ and
$$E\left[\exp\left\{\frac{tX_i}{\sqrt{n}}\right\}\right]=M\left(\frac{t}{\sqrt{n}}\right)$$
Thus,
$$M_{Z_n}(t)=\left[M\left(\frac{t}{\sqrt{n}}\right)\right]^n$$
Let $L(t)=\log M(t)$. Then
\begin{align*} L(0)&=0\\ L'(0)&=\mu=0\\ L^{\prime\prime}(0)&=\sigma^2=1 \end{align*}
Using these values and L’Hôpital’s rule, one can easily show that $nL\left(\frac{t}{\sqrt{n}}\right)\to\frac{t^2}{2}$ as $n\to\infty$, or equivalently, $M_{Z_n}\to e^{\frac{t^2}{2}}$ as $n\to\infty$.
Hence, by the lemma, the central limit theorem is proved when $\mu=0$ and $\sigma^2=1$.

For the general case, let
\begin{align*} Z_n&=\frac{X_1+\cdots+X_n-n\mu}{\sqrt{n}\sigma}\\ &=\frac{\sum_{i=1}^n\frac{X_i-\mu}{\sigma}}{\sqrt{n}},\ n=1,2,\cdots \end{align*}
Let $X_i^\ast=\frac{X_i-\mu}{\sigma}$. Then $Z_n=\frac{\sum_{i=1}^n X_i^\ast}{\sqrt{n}}$ and it can be easily verified that $E[X_i^\ast]=0$ and $\mathrm{var}(X_i^\ast)=1$. For this reason, the $X_i^\ast$’s are called the standardized random variables. Therefore, this completes the proof.

References:

Sheldon Ross, A First Course in Probability, Fifth Edition, Prentice Hall, 1997

Cauchy Random Variable and Which Improper Integral?

The Cauchy random variable $C(m,a)$ with center $m$ and half-width $a$ is defined by the probability density
$$p(x)=\frac{\frac{a}{\pi}}{(x-m)^2+a^2},\ -\infty\leq x\leq\infty$$

Probability density function $p(x)=\frac{\frac{2}{\pi}}{(x-1)^2+2^2}$.


This $p(x)$ can be considered as a probability density since
\begin{align*}\int_{-\infty}^\infty p(x)dx&=\frac{a}{\pi}\int_{-\infty}^\infty\frac{dx}{(x-m)^2+a^2}\\&=\frac{1}{a\pi}\int_{-\infty}^\infty\frac{du}{\left(\frac{u}{a}\right)^2+1}\ (u=x-m)\\&=\frac{1}{\pi}\int_{-\infty}^\infty\frac{dv}{v^2+1}\ \left(v=\frac{u}{a}\right)\\&=\frac{1}{\pi}\lim_{b\to\infty}\left\{\int_{-b}^0\frac{dv}{v^2+1}+\int_0^b\frac{dv}{v^2+1}\right\}\\&=\frac{1}{\pi}\lim_{b\to\infty}\{[\tan^{-1}(v)]_{-b}^0+[\tan^{-1}(v)]_0^b\}\\&=\frac{1}{\pi}\left\{\frac{\pi}{2}+\frac{\pi}{2}\right\}\\&=1\end{align*}
Now, we want to calculate the mean and obviously, we expect it to be $m$.
\begin{align*}\int_{-\infty}^\infty xp(x)dx&=\frac{a}{\pi}\int_{-\infty}^\infty\frac{x}{(x-m)^2+a^2}dx\\
&=\frac{a}{\pi}\int_{-\infty}^\infty\frac{u}{u^2+a^2}du+\frac{am}{\pi}\int_{-\infty}^\infty\frac{du}{u^2+a^2}\ (u=x-m)\\
&=\frac{a}{\pi}\int_{-\infty}^\infty\frac{u}{u^2+a^2}du+m
\end{align*}
But,
$$\lim_{b\to\infty}\int_{-b}^0\frac{u}{u^2+1}du=\frac{1}{2}\lim_{b\to\infty}[\ln(u^2+a^2)]_{-b}^0=-\infty$$ and $$\lim_{b\to\infty}\int_0^b\frac{u}{u^2+1}du=\frac{1}{2}\lim_{b\to\infty}[\ln(u^2+a^2)]_0^b=\infty$$ This means that the mean does not exist! This result does not coincide with our intuition. What about the Cauchy principal value $$\mathrm{p. v.}\int_{-\infty}^\infty xp(x)dx?$$
Before we continue, recall that if $\int_{-\infty}^\infty f(x)dx$ exists (meaning it is finite) then $\mathrm{p. v.}\int_{-\infty}^\infty f(x)dx$ also exist and
$$\int_{-\infty}^\infty f(x)dx=\mathrm{p. v.}\int_{-\infty}^\infty f(x)dx$$
But the converse need not be true as seen below.
$$\mathrm{p. v.}\int_{-\infty}^\infty f(x)dx=\lim_{b\to\infty}\int_{-b}^b\frac{u}{u^2+1}du=0$$
since $\frac{u}{u^2+1}$ is an odd function. Hence, if we choose to use the Cauchy principal value of the improper integral $\frac{a}{\pi}\int_{-\infty}^\infty\frac{u}{u^2+a^2}du$, we obtain the mean $m$ as expected.

Tsiolkovsky Rocket Equation

In this note, we derive the so called Tsiolkovsky rocket equation or simply rocket equation. It is given by
\begin{equation}
\label{eq:rocket}
\Delta v=v_e\ln\frac{m_0}{m_f}=I_{\mathrm{sp}}g_0\ln\frac{m_0}{m_f}
\end{equation}
where

  • $\Delta v$ is the maximum change of velocity of the vehicle;
  • $v_e=I_{\mathrm{sp}}g_0$ is the effective exhaust velocity;
  • $g_0=9.8\ \mathrm{m}/\mathrm{s}^2$ is the gravitational acceleration of an object in a vacuum near the surface of the Earth;
  • $m_0$, called wet mass, is the initial mass, including propellant;
  • $m_f$, called dry mass, is the final total mass without propellant.

The equation \eqref{eq:rocket} is named after the Russian scientist Konstantin Eduardovich Tsiolkovsky (September 5, 1857 – September 19, 1935). He is dubbed the father of Russian rocket science. It is also called fuel equation.

By the Newton’s second law of motion, the net external force $\vec{F}$ to the change in linear momentum $\vec{P}$ of the whole system (including rocket and exhaust) is
$$\vec{F}=\frac{d\vec{P}}{dt}=\lim_{\Delta t\to 0}\frac{\Delta\vec{P}}{\Delta t}$$
$\Delta\vec{P}=\vec{P}_2-\vec{P}_1$, where $\vec{P}_1=m\vec{V}$ is the momentum of the rocket at time $t=0$ and $\vec{P}_2=(m-\Delta m)(\vec{V}+\Delta\vec{V})+\Delta m\vec{V}_e$ is the momentum of the rocket and exhausted mass at $t=\Delta t$. Here, with respect to the observer, $\vec{V}$ is the velocity of the rocket at time $t=0$, $\vec{V}$ is the velocity of the rocket at time $t=\Delta t$, $\vec{V}_e$ is the velocity of the mass added to the exhaust and lost by the rocket during tim $\Delta t$, $m$ is the mass of the rocket at time $t=0$, and $m-\Delta m$ is the mass of the rocket at time $t=\Delta t$. The velocity of the exhaust $\vec{V}_e$ in the observer frame is related to the velocity of the exhaust in the rocket $\vec{v}_e$ by $$\vec{v}_e=\vec{V}_e-\vec{V}$$ or $$\vec{V}_e=\vec{V}+\vec{v}_e$$ Now, $\Delta\vec{P}$ can be written as $$\Delta\vec{P}=m\Delta\vec{V}+\vec{v}_e\Delta m-\Delta m\Delta\vec{V}$$ Since $\Delta m\to 0$ as $\Delta t\to 0$, we have \begin{equation}\label{eq:rocket2}\vec{F}=m\frac{d\vec{V}}{dt}+\vec{v}_e\frac{dm}{dt}\end{equation} If there are no external forces, then $\vec{F}=0$ i.e. $\frac{d\vec{P}}{dt}=0$ (conservation of linear momentum). \eqref{eq:rocket2} then becomes the separable differential equation \begin{equation}\label{eq:rocket3}-m\frac{d\vec{V}}{dt}=\vec{v}_e\frac{dm}{dt}\end{equation} Assuming that $\vec{v}_e$ is constant (Tsiolkovsky’s hypothesis) $v_e$, and integrating \eqref{eq:rocket3} we have $$\int_v^{v+\Delta v}dv=-v_e\int_{m_0}^{m_f}\frac{dm}{m}$$
where $v=|\vec{V}|$, $\Delta v=|\Delta\vec{V}|$, $m_0$ is the initial total mass and $m_f$ is the final mass. Finally, evaluating the integral yields the rocket equation \eqref{eq:rocket}.

From \eqref{eq:rocket}, we obtain
\begin{equation}
\label{eq:rocket4}
\frac{m_0-m_f}{m_0}=1-\frac{m_f}{m_0}=1-e^{-\frac{\Delta v}{v_e}}
\end{equation}
The equation \eqref{eq:rocket4} gives rise to the percentage of the initial total mass which has to be propellant. This tells us how efficient the rocket engine is as shown in the following example.

Example. Let us consider an SSTO (Single-Stage-To-Orbit) rocket. (Most rockets we are seeing are two-stage-to-orbit or three-stage-to-orbit ones.) The rocket uses liquid hydrogen/liquid oxygen for its propellant, so specific impulse is about $I_{\mathrm{sp}}=350$ s. The exhaust velocity is then given by $v_e=3.43$ km/s. $\Delta v$ needed to get the rocket to a 322 km high LEO (Low Earth Orbit) is 8 km/s. With these values \eqref{eq:rocket4} is evaluated to be
$$1-e^{-\frac{\Delta v}{v_e}}=0.9$$
This means that 90% of the initial total mass has to be propellant. The remaining 10% is for the engines, the fuel tank, and the payload. The payload would account for only about 1% of the initial total mass. This kind of rocket is obviously very inefficient and expensive.

In the Sci-Fi novella The Wandering Earth by Liu Cixin (there is also a movie of the same title on Netflix), the Sun will soon become a supernova and facing the ultimate cataclysmic extinction event, people on Earth turns their entire planet into a spaceship and attempt to relocate it to Proxima Centauri which is the closest star to the Sun (about 4.2 light-years). This is an extremely bold idea even in Chinese scale. (Well, they built the Great Wall!) Disappointingly though, in here, I showed using the rocket equation that it is not even possible for startship Earth to break away from its orbit around the Sun.

Counting and Combinations: Permutations and Combinatorics

Let us begin with the following example.

Example. In how many ways can 8 horses finish in a race? Here, we assume that there are no ties.

Solution. $8\times 7\times 6\times 5\times 4\times 3\times 2\times 1=40,320$ ways.

The example above shows an ordered arrangement. Such an ordered arrangement is called a permutation.

Definition. $n$ factorial $n!$ is defined by
\begin{align*} n!&=n(n-1)(n-2)\cdots 3\cdot 2\cdot 1,\ n\geq 1\\ 0!&=1 \end{align*}

The number of permutation of $n$ objects is $n!$.

Example. There are $6!$ permutations of the six letters of the word “square”. In how may of them is $r$ the second letter?

Solution. $5\times 1\times 4!=5!=120$.

Example. Five different books are on a shelf. In how many different ways can you arrange them?

Solution. $5!=120$.

We now consider the permutation of a set of objects taken from a larger set. Suppose we have $n$ items. The number of ordered arrangements of $k$ items from the $n$ items can be then found by
$$n(n-1)(n-2)\cdots (n-k+1)=\frac{n!}{(n-k)!}$$
Denote this by ${}_nP_k$.

Example. How many license plates are there that starts with three letters followed by 4 digits (no repetitions)?

Solution. \begin{align*} {}_{26}P_3\times {}_{10}P_4&=\frac{26!}{23!}\times\frac{10!}{6!}\\
&=26\cdot 25\cdot 24\cdot 10\cdot 9\cdot 8\cdot 7\\
&=78,624,000
\end{align*}

Example. How many five-digit zip codes can be made where all digits are different? The possible digits are 0-9.

Solution. ${}_{10}P_5=\frac{10!}{5!}=30,240$.

Circular permutations are ordered arrangements of objects in a circle. Suppose that circular permutations such as

are considered as different. Under this assumption, let us consider seating $n$ different objects in a circle. There are $n!$ ways to arrange $n$ seats in a circle depending on where we start and there are $n$ different ways to start seating, so there are $\frac{n!}{n}=(n-1)!$ circular permutations. If the orientation does not matter, i.e. if we consider clockwise and counterclockwise directions are the same, there are $\frac{(n-1)!}{2}$ circular permutations.

Example. In how many ways can you seat 6 persons at a circular dinner table?

Solution. $(6-1)!=5!=120$.

Suppose that we are interested in counting the number of ways to choose $k$ objects from $n$ distinct objects without regard to order. It can be calculated as
$$\frac{{}_nP_k}{k!}=\frac{n!}{k!(n-k)!}$$
This is denoted by ${}_nC_k$ or $\begin{pmatrix}n\\k\end{pmatrix}$ and is read “$n$ choose $k$”.

Example. From a group of 5 women and 7 men, how many different committees consisting of 2 women and 3 men can be formed? What if two of the men are feuding and refuse to serve on the committee together?

Solution. For the first question, there are
\begin{align*} {}_5C_2\times {}_7C_3&=\frac{5!}{2!3!}\times\frac{7!}{3!4!}\\ &=350 \end{align*}
such committees. For the second question, the number of ways to two feuding men and another man is ${}_2C_2\times {}_5C_1=5$. So, the number of selecting male committee members that do not include the two feuding men together is ${}_7C_3-5=30$. Consequently, the number of possible committees in this case is ${}_5C_2\times 30=300$.

Theorem. Suppose that $n$ and $k$ are integers such that $0\leq k\leq n$. Then

  1. ${}_nC_0={}_nC_n=1$ and ${}_nC_1={}_nC_{n-1}=n$.
  2. ${}_nC_k={}_nC_{n-k}$.
  3. Pascal’s identity: ${}_{n+1}C_k={}_nC_{k-1}+{}_nC_k$.

Proof. 2. $${}_nC_{n-k}=\frac{n!}{(n-k)!(n-(n-k))!}=\frac{n!}{(n-k)!k!}={}_nC_k$$

  1. \begin{align*} {}_nC_{k-1}+{}_nC_k&=\frac{n!}{(k-1)!(n-k+1)!}+\frac{n!}{k!(n-k)!}\\ &=\frac{n!k}{k!(n+1-k)!}+\frac{n!(n+1-k)}{k!(n+1-k)!}\\ &=\frac{(n+1)!}{k!(n+1-k)!}\\ &={}_{n+1}C_k
    \end{align*}
    The Pascal’s identity allows us to construct the Pascal’s triangle.
    $$\begin{array}{cccccccccc}
    n & & & & & & & & &\\
    0 & & & & & 1 & & & &\\
    1 & & & & 1 & & 1 & & &\\
    2 & & & 1 & & 2 & & 1 & &\\
    3 & & 1 & & 3 & & 3 & & 1 &\\
    4 & 1 & & 4 & & 6 & & 4 & & 1
    \end{array}$$

Example. The chess club has six members. In how many ways

  1. can all six members line up for a picture?
  2. can they choose a president and a secretary?
  3. can they choose three members to attend a regional tournament with no regard to order?

Solution.

  1. ${}_6P_6=6!=720$.
  2. ${}_6P_2=\frac{6!}{4!}=30$.
  3. ${}_6C_3=\frac{6!}{3!3!}=20$.

Theorem. (Binomial Theorem) Let $n$ be a nonnegative integer. Then
$$(x+y)^n=\sum_{k=0}^n{}_nC_k x^{n-k}y^k$$

Proof. We prove it by induction on $n$. For $n=0$,
$$(x+y)^0=1=\sum_{k=0}^0 {}_0C_k x^{-k}y^k$$ For $n=1$, $$\sum_{k=0}^1 {}_1C_k x^{1-k}y^k=x+y$$ Suppose the statement is true up to $n$, i.e. $$(x+y)^n=\sum_{k=0}^n{}_nC_k x^{n-k}y^k$$ \begin{align*} (x+y)^{n+1}&=(x+y)^n(x+y)\\ &=\sum_{k=0}^n {}_nC_k x^{n-k}y^k\\ &=\sum_{k=0}^n {}_nC_kx^{n+1-k}y^k+\sum_{k=0}^n {}nC_k x^{n-k}y^{k+1}\\ &=\sum_{k=0}^n {}_nC_kx^{n+1-k}y^k+\sum_{k=1}^{n+1} {}_nC{k-1} x^{n+1-k}y^k\ (\mbox{replaing $k$ by $k-1$ in the second sum})\\ &={}_nC_0x^{n+1}+\sum_{k=1}^n[{}_nC_k+{}_nC{k-1}]x^{n+1-k}y^k+{}_nC_n y^{n+1}\\ &={}_{n+1}C_0 x^{n+1}+\sum_{k=1}^n {}_{n+1}C_k x^{n+1-k}y^k+{}_{n+1}C_{n+1} y^{n+1}\ (\mbox{Pascal’s identity})\\
&=\sum_{k=0}^{n+1}{}_{n+1}C_k x^{n+1-k}y^k
\end{align*}
This completes the proof.

Example. Expand $(x+y)^6$ using the Binomial Theorem.

Solution.
$$(x+y)^6=x^6+6x^5y+15x^4y^2+20x^3y^3+15x^2y^4+6xy^5+y^6$$

Example. How many subsets are there of a set with $n$ elements?

Solution. There are ${}nC_k$ subsets of $k$ elements, $0\leq k\leq n$. Hence, the total number of subsets of a set of $n$ elements is $$\sum_{k=0}^n {}_nC_k=(1+1)^n=2^n$$

References:

Marcel B. Finan, A Probability Course for the Actuaries

Sheldon Ross, A First Course in Probability, Fifth Edition, Prentice Hall, 1997