It can be easily shown that
$${\rm SO}(2)=\left\{\left(\begin{array}{cc}
\cos\theta & -\sin\theta\\
\sin\theta & \cos\theta
\end{array}
\right): \theta\in[0,2\pi)\right\}\cong{\rm S}^1=\{e^{i\theta}:
\theta\in[0,2\pi)\}.$$Let $\gamma(t)=\left(\begin{array}{cc}
\cos\theta(t) & -\sin\theta(t)\\
\sin\theta(t) & \cos\theta(t)
\end{array}
\right)\in\mathrm{SO}(2)$ with $\theta(0)=0$ and $\dot\theta(0)\ne 0$. Then $\gamma(t)$ be a differentiable (regular) curve in ${\rm SO}(2)$ such that
$\gamma(0)=I$. Thus
$$\dot{\gamma}(0)=\left(\begin{array}{cc}
0 & -1\\
1 & 0
\end{array}\right)\left(\frac{d\theta}{dt}\right)_{t=0}$$
is a tangent vector to $\mathrm{SO}(2)$ at the identity $I$. Hence, the tangent space of ${\rm SO}(2)$ at $I$ is a line i.e. ${\rm SO}(2)$ is a one-dimensional Lie group. (We already know that ${\rm SO}(2)$ is a one-dimensional Lie group since it is identified with the unit circle ${\rm S}^1$.)

Remark. $\dot\gamma(0)=\left(\begin{array}{cc}
0 & -1\\
1 & 0
\end{array}\right)$ is a skew-symmetric matrix, i.e., $\dot\gamma(0)+{}^t\dot\gamma(0)=0$.

Let $\gamma: (-\epsilon,\epsilon)\buildrel{\rm
diff}\over\longrightarrow{\rm O}(n)$ such that $\gamma(0)=I$. Then $\dot{\gamma}(0)$ is a tangent vector to ${\rm O}(n)$ at $I$. Since $\gamma(t)\in{\rm O}(n)$, $$\gamma(t)\cdot{}^t\gamma(t)=I$$ for each $t\in(-\epsilon,\epsilon)$. Thus,
$$\dot{\gamma}(0)\cdot{}^t\gamma(0)+\gamma(0)\cdot\dot{{}^t\gamma}(0)=0.$$ Since ${}^t\gamma(0)=\gamma(0)=I$, $$\dot{\gamma}(0)+\dot{{}^t\gamma}(0)=\dot{\gamma}(0)+{}^t\dot{\gamma}(0)=0.$$ Hence, we see that any tangent vector to ${\rm O}(n)$ at $I$ is represented as a skew-symmetric $n\times n$ matrix. Conversely, we want to show that every skew-symmetric $n\times n$ matrix is a tangent vector to ${\rm O}(n)$ at $I$.

Suppose that $A$ is a $n\times n$ skew-symmetric matrix. As discussed here,
$$e^{At}=I+At+\frac{(At)^2}{2!}+\cdots+\frac{(At)^n}{n!}+\cdots=I+At+\frac{A^2}
{2!}t^2+\cdots+\frac{A^n}{n!}t^n+\cdots$$
is an $n\times n$ matrix.

If $AB=BA$, then by Cauchy’s Theorem,
$$\left(\sum_{k=0}^\infty\frac{A^k}{k!}\right)\left(\sum_{l=0}^\infty\frac{B^l}
{l!}\right)=\sum_{m=0}^\infty\sum_{p=0}^m\frac{A^{m-p}B^p}{(m-p)!p!}=\sum_{m=0}^\infty\frac{(A+B)^m}{m!}.$$ This implies that $e^Ae^B=e^{A+B}$ if $AB=BA$. In particular, $e^{A}e^{-A}=e^0=I$ so that $e^A$ is non-singular. If $A$ is skew-symmetric, then ${}^t(e^{At})=e^{{}^tAt}=e^{-At}$ and so $e^{At}\cdot{}^t(e^{At})=I$, i.e., $e^{At}\in{\rm O}(n)$. Now, $\displaystyle\frac{de^{At}}{dt}=Ae^{At}$ and $\dot{e^{At}}(0)=A$, i.e., the skew-symmetric matrix $A$ is a tangent vector to ${\rm O}(n)$ at $I$.

Proposition. The tangent space of ${\rm O}(n)$ or ${\rm SO}(n)$ at $I$ is the set of all $n\times n$ skew-symmetric matrices. Denote by ${\mathfrak o}(n)$ (${\mathfrak s\mathfrak o}(n)$) the tangent space of ${\rm O}(n)$ (${\rm SO}(n)$, respectively) at $I$. Note that $\dim{\mathfrak o}(n)=\displaystyle\frac{1}{2}n(n-1)$. This can be easily shown.

Definition. The tangent space ${\mathfrak o}(n)$ (${\mathfrak s\mathfrak o}(n)$) to the Lie group ${\rm O}(n)$ (${\rm SO}(n)$, respectively) at $I$ is called the Lie algebra of ${\rm O}(n)$ (${\rm SO}(n)$, respectively).