Let $\gamma: [0,1]\longrightarrow M$ be a path. Using connection $\nabla$, one can consider the notion of moving a vector in $L_{\gamma(0)}$ to $L_{\gamma(1)}$ without changing it. This is parallel transporting a vector from $L_{\gamma(0)}$ to $L_{\gamma(1)}$. The change is measured relative to $\nabla$, so if $\xi(t)\in L_{\gamma(t)}$ is moving without changing, it must satisfy the differential equation
$$\nabla_{\dot{\gamma}(t)}\xi=0,$$
where $\dot{\gamma}(t)$ is the tangent vector field to the curve $\gamma(t)$. The image of $\gamma(t)$ is covered by the $U_\alpha$’s on which $L$ has nowhere vanishing sections $s_\alpha$’s. Since $\gamma([0,1])$ is compact, the image of $\gamma$ is covered by only finitely many of such open sets. Let $U_\alpha$ be one of such open sets and assume that it contains $\gamma(0)$. Then $\xi|_{U_\alpha}(t)=\xi_\alpha(\gamma(t))s_\alpha(\gamma(t))$ where $\xi_\alpha: U_\alpha\longrightarrow\mathbb{C}$.
\begin{align*}
\nabla_{\dot{\gamma}(t)}\xi&=d\xi_\alpha(\dot{\gamma}(t))s_\alpha(\gamma(t))+A_\alpha(\dot{\gamma}(t))\xi_\alpha(\dot{\gamma}(t))s_\alpha(\gamma(t))\\
&=\left(\frac{d\xi_\alpha}{dt}(\gamma(t))+A_\alpha(\dot{\gamma}(t))\xi_\alpha(\dot{\gamma}(t))\right)s_\alpha(\gamma(t)).
\end{align*}
$\nabla_{\dot{\gamma}(t)}\xi=0$ implies that
$$\frac{d\xi_\alpha}{dt}=-A_\alpha(\dot{\gamma}(t))\xi_\alpha.$$
The solution of this equation is given by
$$\xi_\alpha(t)=\xi_\alpha(\gamma(0))\exp\left(-\int_0^tA_\alpha(\dot{\gamma}(u))du\right).$$
The standard existence and uniqueness theorems (Frobenius’ theorem) tell that parallel transport defines an isomorphism $L_{\gamma(0)}\cong L_{\gamma(t)}$ for any $\gamma(t)\in U_\alpha$. Suppose that the path $\gamma$ is covered by finitely many open sets $U_\alpha$, $U_{\alpha_1}$, $U_{\alpha_2}$, $\cdots$, $U_{\alpha_n}$ as shown in the following figure.

As discussed, we know that $L_{\gamma(0)}\cong L_{\gamma(t_1)}$. Using $\xi_{\alpha_1}(\gamma(t_1))=\xi_\alpha(\gamma(t_1))$ as the initial condition, we also find $\xi|_{U_{\alpha_1}}(t)=\xi_{\alpha_1}(\gamma(t))s_{\alpha_1}(\gamma(t))$ where
$$\xi_{\alpha_1}(t)=\xi_{\alpha_1}(\gamma(t_1))\exp\left(-\int_{t_1}^tA_{\alpha_1}(\dot{\gamma}(u))du\right).$$
This implies that  $L_{\gamma(t_1)}\cong L_{\gamma(t_2)}$. Continuing this process, we obtain $L_{\gamma(t_2)}\cong L_{\gamma(t_3)}$, $\cdots$, $L_{\gamma(t_n)}\cong L_{\gamma(1)}$. Since the relation $\cong$ is transitive, we have
$$L_{\gamma(0)}\stackrel{P_\gamma}{\cong}L_{\gamma(1)}.$$
In general, $P_\gamma$ depends on $\gamma$ and $\nabla$. Now we are particularly interested in the case when $\gamma:[0,1]\longrightarrow M$ is a loop i.e. $\gamma(0)=\gamma(1)$. Then we can define the holonomy $\mathrm{hol}(\nabla,\gamma)$ of the connection $\nabla$ along the loop $\gamma$ by
$$P_\gamma(s)=\mathrm{hol}(\nabla,\gamma)s$$
for any nowhere vanishing section $s\in L_{\gamma(0)}$. So, what is really the meaning of the holonomy? In Euclidean space (the world we are familiar with), we can move a  vector without changing its direction and magnitude by parallel translation. That is, in Euclidean space parallel translation is parallel transport. So, we do not distinguish vectors that have the same direction and magnitude in Euclidean space. In a curved manifold, there is no such parallel translation and parallel transport is considered relative to the connection $\nabla$ as we discussed above. For those who live in a manifold with connection $\nabla$, they will not know the difference when a vector is parallel transported relative to $\nabla$ along a loop. The initial vector and the one that comes back to the initial point after parallel transport must coincide. However, in our perspective (for those who live in Euclidean space) we notice a difference between them. The holonomy measures such a difference.

Since $\gamma$ is a loop, both $\gamma(0)$ and $\gamma(1)$ belong to the same open set, say $U_\alpha$.
\begin{align*}
P_\gamma(\xi(0))&=\xi(1)\\
&=\xi_\alpha({\gamma(1)})\exp\left(-\oint_{\gamma}A_\alpha(\dot{\gamma}(u))du\right)s(\gamma(1))\\
&=\xi_\alpha({\gamma(0)})\exp\left(-\oint_{\gamma}A_\alpha(\dot{\gamma}(u))du\right)s(\gamma(0)).
\end{align*}
On the other hand, $\xi(0)=\xi_\alpha(\gamma(0))s(\gamma(0))$. So
\begin{align*}
P_\gamma(\xi(0))&=P_\gamma(\xi_\alpha(\gamma(0))s(\gamma(0)))\\
&=\xi_\alpha(\gamma(0))P_\gamma(s(\gamma(0))).
\end{align*}
Hence, we see that
$$P_\gamma(s(\gamma(0)))=\exp\left(-\oint_{\gamma}A_\alpha(\dot{\gamma}(u))du\right)s(\gamma(0))$$
and that the holonomy is given by
$$\mathrm{hol}(\nabla,\gamma)=\exp\left(-\oint_\gamma A_\alpha\right).$$
If $\gamma$ is the boundary of a disk, then by Stokes’ theorem we have
\begin{align*}
\mathrm{hol}(\nabla,\gamma)&=\exp\left(-\int_DdA_\alpha\right)\\
&=\exp\left(-\int_D F\right)\ \ \ \ \ \ \ (1)
\end{align*}
where $D$ is the interior of the disk.

Proposition. If $L\stackrel{\pi}{\longrightarrow}M$ is a line bundle with connection $\nabla$ and $\Sigma$ is a compact submanifold of $M$ with boundary loop $\gamma=\partial M$, then
$$\mathrm{hol}(\nabla,\gamma)=\exp\left(-\int_\Sigma F\right).\ \ \ \ \ \ \ (2)$$

Proof. By compactness, we can triangulate $\Sigma$ so that each of the triangles is in some $U_\alpha$. Then we apply (1) to each triangle and the holonomy up and down the interior edges cancels to give the required result.

Remark. Clearly holonomy is a gauge invariant quantity. In gauge theory, (2) is called a Wilson line or a Wilson loop. It is important to note that the gauge connection may be constructed from the collection of Wilson loops up to gauge transformation.

Example. [Parallel Transport on the 2-Sphere] In this example, we calculate the holonomy of the standard connection on $TS^2$. Before we proceed, let us take look at the figure below.

It clearly shows that the holonomy is $e^{i\theta}$ since the discrepancy between the initial vector and the parallel transported vector along the loop is given by a rotation by angle $\theta$. Recall that
\begin{align*}
\frac{\partial}{\partial\theta}&=(-\sin\theta\sin\phi,\cos\theta\cos\phi,0),\\
\frac{\partial}{\partial\phi}&=(\cos\theta\cos\phi,\sin\theta\cos\phi,-\sin\phi).
\end{align*}
The unit normal vector field $\hat n$ is computed to be
\begin{align*}
\hat n&=(\cos\theta\sin\phi,\sin\theta\sin\phi,\cos\phi)\\
&=\sin\phi\frac{\partial}{\partial\phi}\times\frac{\partial}{\partial\theta}.
\end{align*}
Consider a nowhere vanishing section
$$s=(-\sin\theta,\cos\theta,0).$$
Then
$$ds=(-\cos\theta,-\sin\theta,0)d\theta.$$
\begin{align*}
\nabla s&=\pi(dx)\\
&=ds-\langle ds,\hat n\rangle\hat n\\
&=(-\cos\theta\cos^2\phi,-\sin\theta\cos^2\phi,\sin\theta\cos\phi)d\theta\\
&=\cos\phi\hat n\times s d\theta\\
&=i\cos\phi sd\theta.
\end{align*}
The last expression is obtained by the definition of scalar multiplication
$$(\alpha+i\beta)v=\alpha v+\beta\cdot u\times v$$
for $\alpha,\beta\in\mathbb{C}$ and $u,v\in T_\ast S^2$, as seen here. So the connection 1-form is
$$A=i\cos\phi d\theta$$
and the curvature is
\begin{align*}
F&=dA\\
&=-i\sin\phi d\phi\wedge d\theta.
\end{align*}
Note that $\sin\phi d\phi\wedge d\theta$ is the area form on $S^2$, so
$$F=-i\mathrm{area}.$$
The area of the region bounded by the loop is
$$\int_0^\theta\int_0^{\frac{\pi}{2}}\sin\phi d\phi d\theta=\theta.$$
Therefore, the holomony is
$$\mathrm{hol}(\nabla,\gamma)=e^{i\theta}$$
as we already know.

References:

[1] M. Murray, Notes on Line Bundles