Denote by $\langle\ ,\ \rangle$ the standard Euclidean inner product in $\mathbb{R}^n$. Then for any $v,w\in\mathbb{R}^n$,
$$\langle v,w\rangle={}^tvw.$$
Definition. A bijective map $A: \mathbb{R}^n\longrightarrow\mathbb{R}^n$ is said to be an isometry if it preserves the inner product $\langle\ ,\ \rangle$ i.e. for any vectors $v,w\in\mathbb{R}^n$
$$\langle Av,Aw\rangle=\langle v,w\rangle.$$
Isometries are usually called symmetries in physics.
The set of all isometries of $\mathbb{R}^n$ forms a group with composition $\circ$. The group is denoted by $\mathrm{E}(n)$ and called the Euclidean motion group. The orthogonal group $\mathrm{O}(n)$ is a subgroup of $\mathrm{E}(n)$ and that it is a group of all isometries of $\mathbb{R}^n$ which leaves the origin fixed. In other words,
Theorem. $\mathrm{O}(n)$ is the group of all linear isometries of $\mathbb{R}^n$.
Proof. Let $A\in\mathrm{O}(n)$. Then
\begin{align*}
\langle Av,Aw\rangle&=\langle v,{}^tAAw\rangle\\
&=\langle v,w\rangle
\end{align*}
since ${}^tAA=I$. Hence, $A:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ is an isometry. Conversely, if $A:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ is an isometry, then for any $v,w\in\mathbb{R}^n$
\begin{align*}
\langle Av,Aw\rangle=\langle v,w\rangle&\Longrightarrow\langle{}^tAAv,w\rangle=\langle v,w\rangle\\
&\Longrightarrow\langle {}^tAAv-v,w\rangle=0.
\end{align*}
Since $w$ is arbitrary,
$$({}^tAA-I)v=0.$$
Since $v$ is also arbitrary,
$${}^tAA=I$$
i.e. $A$ is an orthogonal matrix.
Remark. For any $v,w\in\mathbb{R}^n$,
$$||v+w||^2=||v||^2+2\langle v,w\rangle+||w||^2$$
where $||v||=\sqrt{\langle v,v\rangle}$, the Euclidean norm of $v$. Plugging in $Av$ and $Aw$ for $v$ and $w$ respectively, we obtain
$$\langle Av,Aw\rangle=\frac{1}{2}(||A(v+w)||^2-||Av||^2-||Aw||^2).$$
This equation tells that a linear isomorphism $A:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ is an isometry if and only if it preserves the normĀ $||\cdot||$.
Let $A\in\mathrm{O}(n)$. Since $\det A=\det {}^tA$, $\det A=\pm 1$. This implies that $\mathrm{O}(n)$ has two connected components. One that contains orthogonal matrices whose determinant is $1$, i.e. $\mathrm{SO}(n)$, and the other that contains orthogonal matrices whose determinant is $-1$. The identity component $\mathrm{SO}(n)$ of $\mathrm{O}(n)$ is the group of all linear isometries that preserve orientation. Clearly $\mathrm{SO}(n)$ is a normal subgroup of $\mathrm{O}(n)$.
The Lie groups $\mathrm{O}(n)$ and $\mathrm{SO}(n)$ are compact.