This is my code:
ClearAll[f, g] f[x_] := x^3 g[h_, x_] := h[x] - h[x + 1] g[f, x] (* x^3 - (1 + x)^3 *) g[g[f, x], x] (* (x^3 - (1 + x)^3)[x] - (x^3 - (1 + x)^3)[1 + x] *) r[x_] = g[f, x] (* x^3 - (1 + x)^3 *) g[r, x] (* x^3 - 2 (1 + x)^3 + (2 + x)^3 *) g[f,x] outputs what I'd expect. However, if I try g[g[f,x],x], I don't get the expected result.
I think I see why; g[f,x] is producing an equation, whereas I believe I need a function head?
The last two lines gives the correct result ( x^3 - 2 (1 + x)^3 + (2 + x)^3) by defining r[x] as the output of g[f,x] and then calculating g[r,x], but I am looking to reperform this recursively and so this solution doesn't easily work, and in any case is clunky. I'm sure there must be a neat elegant solution to allow me to use something like g[g[f,x],x] or NestList, but I'm not seeing it.