a property of log determinant

First, as I mentioned in my comment, note that $\det X = \exp \operatorname{tr} \ln X$, which means for the function $f$ that

$$f(X)=-\ln \det X = - \operatorname{tr} \ln X$$

It can also be shown that $f'(X)=-X^{-1}$, where one should take a Fréchet derivative.

Therefore,

$$f(X+D) = f(X(1+X^{-1}D))=f(X)+f(1+X^{-1}D)$$

Putting $A=X^{-1}D$ the inequality we have to prove can also be written as:

$$f(1+A)\le -\operatorname{tr}(A)+\frac{\operatorname{tr}(A^2)}{2\left(1-\sqrt{\operatorname{tr}(A^2)}\right)^2} \; .$$

Now, this should be tackled with the Taylor theorem for Fréchet derivable functions. But for now, I'll make the extra assumption that $A$ is diagonalizable with real eigenvalues. In that case, if we call the eigenvalues of $A$ $\alpha_j$. The inequality becomes

$$-\sum_j \ln (1+\alpha_j) \le -\sum_j \alpha_j + \frac{\sum_j \alpha_j^2}{2\left(1-\sqrt{\sum_j \alpha_j^2}\right)^2} \; .$$

This immediately follows from Taylor's theorem since

$$-\ln(1+\alpha_j) = -\alpha_j + \frac{1}{2}\frac{1}{(1+c_j)^2} \alpha_j^2$$

for some $c_j \in ]0,\alpha_j[$ (or $]\alpha_j,0[$ if $\alpha_j<0$ ) . Note also that for the Taylor formula to hold, we need $|\alpha_j|<1$, which is guaranteed by $\sum_j \alpha_j^2<1$.

Now, we don't know the $c_j$, but we know they all are within an interval about $0$ of radius $\sqrt{\sum_j \alpha_j^2}$. For each of these $c_j$ it is true that

$$\frac{1}{(1+c_j)^2} \le \frac{1}{(1-\sqrt{\sum_j \alpha_j^2})^2}$$

and this gives us the inequality.

I'll try to work out the proof from the Fréchet Taylor formula if I have some time.