I recently learnt about dual numbers, of the form $u + v\varepsilon$ where $u, v \in \mathbb R$ and $\varepsilon \notin \mathbb R$ is such that $\epsilon^2 = 0$. These numbers are used for automatic differentiation as they allegedly satisfy $f'(x) = \operatorname{Dual}(f(x+\varepsilon))$, where $\operatorname{Dual}$ is the coefficient of $\varepsilon$ in the cartesian expansion of a dual number. However, all resources I came across just showed this property for a few simple examples, usually polynomial.
I tried to prove or reject this property for a more general class of functions. So let's assume $f : (I \subseteq \mathbb R) \to \mathbb R$ is differentiable and $\delta$ is a small non-zero real number. We can then write $f(x+\delta) = f(x) + \delta f'(x) + \delta o(1)$, where $o(1)$ goes to $0$ as $\delta$ goes to $0$. Assuming we manage to make $\varepsilon$ behave like a small non-zero real, we get $f(x+\epsilon) = f(x) + \varepsilon f'(x) + \varepsilon o(1)$, and if $o(1) = \varepsilon \times \text{something}$ we can conclude that the dual part of $f(x+\varepsilon)$ is indeed $f'(x)$. This happens when $f$ can be expressed as a power series, which already helps cover a nice range of functions.
However:
- being differentiable likely isn't enough to having a first-order Taylor expansion with a remainder of the form $\epsilon^2 \times \text{stuff}$, but can we make weaker assumptions?
- can we even extend real functions to dual reals in a way that guarantees that $f(x+\varepsilon)$ looks like $f(x+\delta)$ for small $\delta$?
As an example for the second point, defining absolute value seems quite non-trivial. Assuming $|x+\varepsilon|$ is a square root of $(x+\varepsilon)^2$, calculations show that it must equal $\pm(x+\epsilon)$ if $x \neq 0$ and $0$ otherwise. To be able to find the derivative of $x \mapsto |x|$ and $x \mapsto x|x|$, we figure out that the best choice is probably to set $|x + \varepsilon| = x+\varepsilon$ if $x > 0$, $-(x+\varepsilon)$ if $x < 0$ and $0$ if $x = 0$.
