To find the limit of three terms mean iteration

I find this a very nice and natural idea for generalizing the usual AGM to three or more variables, by updating the $k$-th variable as $k$-th root of the average of the terms of the $k$-th elementary symmetric polynomial, see below.

It looks that natural to me that I'd be surprised this isn't known yet, but I never saw it before. I'd suggest to call this simply AGM(a,b,c) or AGM($x_1, ..., x_m$) in the general case, i.e., call it the arithmetic-geometric mean of the m values, to answer the first question, "how it can be named?"

As to the next question, yes, we can show that $(a_n),~(b_n),~(c_n)$ all converge to a common limit, assuming that the initial values are nonnegative to ensure the roots are well defined: Assume that $a\ge b\ge c$, without loss of generality because the three are symmetrically "mixed" right at the first step. Then these inequalities will be preserved after each update, and more precisely we will have $$a_n \ge a_{n+1} \ge b_{n+1} \ge c_{n+1} \ge c_{n}$$ for all $n$, with equalities if and only if all numbers are equal. This is proved exactly like the usual Inequality of arithmetic and geometric means. (For the "intermediate terms" between the extremal cases of arithmetic mean (first variable) and geometric mean (last variable) one may apply corresponding AGM-inequalities to pairs of variables.)

From this inequality, by the monotone convergence theorem it follows that all of the sequences $(a_n)$, $(b_n)$ and $(c_n)$ converge. The limits $a^*$, $b^*$, $c^*$ must also satisfy the above inequalities, and must be equal. (If $\delta:=a^* - c^*>0$, then $a_n-a_{n+1}\ge\delta/3$ by definition of the arithmetic mean, contradicting convergence.) ■

Now the most beautiful part: The above easily generalizes to an arbitrary number of $m\ge 1$ variables. To make more explicit what I said in the first phrase, for $\mathbf x=(x_1,...,x_m)\in\mathbb R_+^m$, let $ AGM(\mathbf x) = \lim\limits_{n\to\infty}x_1^{(n)}$ (or equivalently any other index $k$), with $\mathbf x^{(0)} = \mathbf x$ and $ x_k^{(n+1)} = \big(f_k^{\,}(\mathbf x^{(n)})\big) ^{1/k}$, $1\le k\le m$, $n\ge 0$, where $$ f_k^{\,}(\mathbf x)={m\choose k}^{-1}\sum_{1\le j_1<...<j_k\le m} x_{j_1}\cdots x_{j_k} $$ is the $k$-th elementary symmetric polynomial in $m$ variables, normalized by the number ${m\choose k}=m!/(k!(m-k)!)$ of terms in that polynomial. For $m=1$ this gives $AGM(x)=x$, while for $m=2$ it gives the usual AGM (with $f_1(x)=(x_1+x_2)/2$ and $f_2(x)=x_1\,x_2$), and for $m=3$ it gives the sequence defined in the original post/question.