Let $A$ be a Hermitian (self-adjoint) operator. Due to Max Born, who proposed the statistical interpretation of quantum mechanics, the probability of measuring an eigenvalue $\lambda_i$ of $A$ in a state $\psi$ is $\langle\psi|P_i|\psi\rangle$, where $P_i$ is the projection onto the eigenspace of $A$ corresponding to $\lambda_i$ i.e. $P_i$ is a linear map $P_i: V_{\lambda_i}\longrightarrow V_{\lambda_i}$ such that $P^2=I$. If we assume no degeneracy of $\lambda_i$, then the eigenspace of $A$ corresponding to $\lambda_i$ is one-dimensional. In this case, $P_i=|\lambda_i\rangle\langle\lambda_i|$, where the $|\lambda_i\rangle$ are orthonormal. Now,
\begin{align*} \langle\psi|P_i|\psi\rangle&=\langle\psi|\lambda_i\rangle\langle\lambda_i|\psi\rangle\\ &=\overline{\langle\lambda_i|\psi\rangle}\langle\lambda_i|\psi\rangle\\ &=|\langle\lambda_i|\psi\rangle|^2 \end{align*}
The complex number $\langle\lambda_i|\psi\rangle$ is called the probability amplitude, so the probability is the squared magnitude of the amplitude. This is called the Born rule. (It is named after Max Born.) Suppose $|\psi\rangle$ is a unit vector. Then since $\sum_i|\langle\lambda_i|\psi\rangle|^2=1$, we have \begin{equation}\label{eq:complete}\sum_iP_i=\sum_i|\lambda_i\rangle\langle\lambda_i|=I\end{equation} \eqref{eq:complete} is called the completeness of the $|\lambda_i\rangle$.

One may consider $A$ as a random variable with its eigenvalues as the values of random variable. (See Remark below.) Since for each $j$, $A|\lambda_j\rangle=\lambda_j|\lambda_j\rangle$,
$$\langle\lambda_i|A|\lambda_j\rangle=\lambda_j\langle\lambda_i|\lambda_j\rangle=\lambda_j\delta_{ij}$$
The expected value $\langle A\rangle$ of the self-adjoint operator $A$ in the state $|\psi\rangle$ is naturally defined as a weighted average
\begin{align*} \langle A\rangle&=\sum_i\lambda_i|\langle\lambda_i|\psi\rangle|^2\\ &=\sum_i\langle\lambda_i|\psi\rangle\langle\psi|\lambda_i|\lambda_i\rangle\\
&=\sum_i\langle\lambda_i|\psi\rangle\langle\psi|A|\lambda_i\rangle\ (A|\lambda_i\rangle=\lambda_i|\lambda_i\rangle)\\
&=\sum_i\langle\psi|A|\lambda_i\rangle\langle\lambda_i|\psi\rangle\\&=\langle\psi|A|\sum_i|\lambda_i\rangle\langle\lambda_i|\psi\rangle\\&=\langle\psi|A|I|\psi\rangle\ (\mbox{by the completeness of the}\ |\lambda_i\rangle)\\&=\langle\psi|A|\psi\rangle \end{align*}
Hence, we have
$$\langle A\rangle=\langle\psi|A|\psi\rangle$$

Remark. Let $\Omega=\bigcup_iV_{\lambda_i}$, i.e. $\Omega$ is the set of all eigenvectors of $A$. Let $\mathcal{U}$ consist of $\emptyset$, $\Omega$ and unions of subsequences of ${V_{\lambda_i}}$. Then $\mathcal{U}$ is a $\sigma$-algebra of subsets of $\Omega$. Define $X:\Omega\longrightarrow\mathbb{R}$ by
$$X|\lambda_i\rangle=\langle\lambda_i|A|\lambda_i\rangle=\lambda_i$$
Then $X$ is a random variable.