Limit of double sum is always series sum?

Without assuming absolute convergence, the left and the right side may converge do different values or diverge independently. For example, let

$$b_{i,0} = 1$$ $$\forall j > 0, b_{i,j} = \frac 1 {j + 1} - \frac 1 {j}$$

where $i$ may take arbitrary values. In this case, $\sum_{j=0}^{n} b_{i,j} = \frac 1 {n + 1}$, for all $n > 0$.

The left side becomes

$$ \lim_{n \to + \infty} \left( \sum_{i=0}^{n} \sum_{j=0}^{n} b_{i, j} \right) = \lim_{n \to + \infty} \left( n \frac 1 {n + 1} \right) = 1 $$

while the right one becomes

$$ \sum_{i=0}^{+ \infty} \left( \sum_{j=0}^{+ \infty} b_{i, j} \right) = \sum_{i=0}^{+ \infty} \left( \lim_{n \to + \infty} \frac 1 {n + 1} \right) = 0 $$

moreover, if we'd reverse the order of the summation symbols, we'd have some pretty messy situation.

---

Conversely, if we know that either limit exists for the absolute values of the elements, then this, and virtually any such equality, should hold. More precisely, let $X$ be a normed vector space, say $\mathbb R$ or $\mathbb C$:

1. Suppose $\sigma: \mathbb N \to \mathbb N$ is bijective (an "infinite transposition" of the indices). And that the sequence $a: \mathbb N \to X$ is converging absolutely:

$$\sum_{k=0}^{\infty} |a_k| \in \mathbb R_{\geq 0}$$

then

$$\sum_{k=0}^{\infty} a_k = \sum_{k=0}^{\infty} a_{\sigma(k)}$$

2. And also, if $\tau: \mathbb N \to \mathbb N \times \mathbb N$ is another bijection, arranging the sequence of indices into an "infinite matrix" and vice versa, and $b: \mathbb N \times \mathbb N \to X$ is such a matrix. Then

$$\sum_{k=0}^{\infty} |b_{\tau(k)}| \in \mathbb R_{\geq 0}$$

or

$$\sum_{i=0}^{\infty} \sum_{j=0}^{\infty} |b_{i, j}| \in \mathbb R_{\geq 0}$$

implies

$$\sum_{i=0}^{\infty} \sum_{j=0}^{\infty} b_{i, j} = \sum_{k=0}^{\infty} b_{\tau(k)}$$

A proof goes like this:

1. First consider the case of a positive series, and establish that the sum of original (on $k$) series is $\leq$ the sum of the transposed series (on $\sigma(k)$). Since the former series may also be viewed as transposition of the latter, we also have the opposite inequality -- i.e. the sums are equal.

   The extrapolation onto the general case of arbitrary such series is done by virtue of grouping the elements of the same sign, i.e. representation of the series as double sum: first by the elements having the same sign, then by the resulting values applied to the corresponding signs. (e.g. in case of $X = \mathbb R$, it just two signs, $+$ and $-$)

2. In one direction, assuming $\bar{B} := \sum_{k=0}^{\infty} |b_{\tau(k)}|$ is defined.

Then $\bar{B}$ is an upper bound for the increasing sequences of partial sums:

$$ \bar{B}'_i(m) := \sum_{j = 0}^{j = m} |b_{i,j}| \leq \bar{B} $$

so these sums are converging:

$$ \sum_{j = 0}^{\infty} |b_{i,j}| \text{ and } \sum_{j = 0}^{\infty} b_{i,j} =: B'_i $$

and we want yet to check that

$$ \sum_{i = 0}^{\infty} B'_i = B := \sum_{k = 0}^{\infty} b_{\tau(k)} $$

Due to the absolute convergence, for any $\epsilon > 0$, we have

$$ \left| B - \sum_{k = 0}^{k = \phi(\epsilon)} b_{\tau(k)} \right| = \left| \sum_{k = \phi(\epsilon) + 1}^{\infty} b_{\tau(k)} \right| \leq \sum_{k = \phi(\epsilon) + 1}^{\infty} |b_{\tau(k)}| \lt \epsilon $$

where $\phi: \mathbb R_{\gt 0} \to \mathbb N$ is the "estimate map", from the definition of the limit, for the absolute convergence for $b_{\tau(k)}$.

And if $S$ is an arbitrary subset of indices, such that $[0, \phi(\epsilon)] \subseteq S \subseteq \mathbb N$, then also

$$ \left| B - \sum_{k \in S} b_{\tau(k)} \right| = \left| \sum_{k \in \mathbb N \setminus S} b_{\tau(k)} \right| \leq \sum_{k \in \mathbb N \setminus S} |b_{\tau(k)}| \leq \sum_{k = \phi(\epsilon) + 1}^{\infty} |b_{\tau(k)}| \lt \epsilon $$

generally, if $S$ and $S'$ are two such sets, and $S \subseteq S'$, then $\left| B - \sum_{k \in S'} b_{\tau(k)} \right| \leq \left| B - \sum_{k \in S } b_{\tau(k)} \right|$.

Let $\psi (f, k) := \max_{0 \le x \le k} f(x)$, and $(m, n) := \psi (\tau, \phi (\epsilon))$ defines the "index rectangle", based at $(0, 0)$ and containing all the values $\tau(0), \dots, \tau(\phi(\epsilon))$. We define $S$ to be $\tau^{-1}([0, m], [0, n])$, i.e. a set of such an indices, that are mapped by $\tau$ into the rectangle (large enough to satisfy the requirements on $S$). Then

$$ \left| B - \sum_{k \in S} b_{\tau(k)} \right| = \left| B - \sum_{i = 0}^{i = m} \sum_{j = 0}^{j = n} b_{i,j} \right| $$

and, since all such expressions are bound by $\epsilon$, and may only get smaller with $m$ and $n$, we also have

$$ \left| B - \sum_{i = 0}^{\infty} \sum_{j = 0}^{\infty} b_{i,j} \right| < \epsilon $$

Since $\epsilon$ may be arbitrarily small, necessary

$$ \sum_{i = 0}^{\infty} \sum_{j = 0}^{\infty} b_{i,j} = B $$

In the other direction, assuming $\bar{B} := \sum_{i=0}^{\infty} \sum_{j=0}^{\infty} |b_{i, j}|$ is defined. Then

$$ \sum_{i=0}^{i=m} \sum_{j=0}^{j=n} |b_{i, j}| \leq \bar{B} $$

and

$$ \sum_{k=0}^{k=l} |b_{\tau(k)}| \leq \sum_{i=0}^{i=m} \sum_{j=0}^{j=n} |b_{i, j}| $$

where as before $(m, n) := \psi (\tau, l)$.

So $l \mapsto \sum_{k=0}^{k=l} |b_{\tau(k)}|$, the sequence of positive partial sums, is increasing and bounded, therefore, converging.

The equality of the limits follows from the previous result.

$\square$

Combination of these two rules gives the answer to the (second) question. And, among other things, it allows to swap the summation symbols for absolutely converging series. These rules may be viewed as a generalization of the commutativity of addition.

---

See also: https://en.wikipedia.org/wiki/Series_(mathematics)