Why is white noise uncorrelated for any two different samples?

You are correct in pointing out that the definition of the Dirac delta is not that it equals zero at $\tau \neq 0$ and infinity at $\tau = 0$, but rather that it integrates to one over its entire range and effectively "samples" the function it is multiplied with at $\tau = 0$ when integrated, giving the notation \begin{equation} \int\delta(\tau)f(\tau)d\tau=f(0) \end{equation}

Another notation that I find helpful is to view the delta function from a centered normal distribution \begin{equation} \delta(x) \approx \lim_{\sigma\rightarrow 0} \frac{1}{\sqrt{2\pi\sigma^{2}}}e^{-\frac{x^{2}}{2\sigma^{2}}} \end{equation} This approximation works since the integral of any PDF is \begin{equation} \int_{-\infty}^{\infty}f(x)dx = 1 \end{equation}

Practically speaking, in terms of proving that $x(t)$ is uncorrelated with $x(t+\tau)$, if you integrate $r_{x}(\tau)$ with any other function $f(\tau)$, you would get \begin{equation} \int r_{x}(\tau)f(\tau)d\tau = \int \frac{N_{0}}{2}\delta(\tau)f(\tau)d\tau = \frac{N_{0}}{2}f(0) \end{equation} thereby showing that $r_{x}(\tau)$ contributes a non-zero value only at $\tau = 0$, which shows that $x(t)$ is only correlated with a non-shifted version of itself.

If you want to delve further into the Dirac delta, you would have to delve into measure theory. However, further exploration into this topic is better asked on the Math SE, and I would not be qualified to give you that answer.

Well, the usefulness in this notation lies in the fact that you can use it to show that when multiplied-then-integrated with another function, you get that other function.

To see that, try this:

1. Assume an energy-bound function $g(\tau)$. (If it helps imagination, this is the time-reversed complex conjugate channel impulse response $h^*(-\tau)$; so, instead of convolving with $h(\tau)$ to model what happens to our noise as it goes through a system, we just correlate with $g(\tau)$. Same operation, different notation.)
2. correlate your (random) white noise $X(t)$ with that to get a (random) signal $Y(t)$
3. Write down the cross-corr $R_{XY}$; be sure to use different integration variables, so that you can see that you can "nest" these integrals
4. (optional but very helpful:) Close your eyes and say "Fubini is my friend, fear is the mind-killer, fear is the little death", as you
5. swap the order of the integrals: you get something like $$R_{XY}=\int R_{XX}(u)\cdot g(u) \,\mathrm du.$$
6. realize that if that autocorrelation $R_{XX}$ really has the properties of the Dirac delta, then you just invented a system identification method (a channel sounder, in our example from 1.): excite a linear, memory-having system with white noise and correlate its output with the same white noise; on expectation (and assuming ergodicity, thus after arbitrarily long sample correlation) you get the system impulse response out.