Both results indicate the both integrals result in zero.
The integrand is dominated by the part `Cosh[(ArcSinh[a] - ArcSinh[b])/2]*Exp[-1/2*(a^2 + b^2)] `
Plot3D[ Cosh[(ArcSinh[a] - ArcSinh[b])/2]*Exp[-1/2*(a^2 + b^2)]
, {a, -10 , +10 }, {b, -10 , +10 },
PlotPoints -> 100, PlotRange -> All, AxesLabel -> {a, b}]
[![enter image description here][1]][1]
For numerical reasons it is sufficient to decrease the integration range accordingly.
inf=10;
f[x_, y_] :=NIntegrate[
Cosh[(ArcSinh[a] - ArcSinh[b])/2]*Exp[-1/2*(a^2 + b^2)]*
Cos[x*(a - b) +y*(Sqrt[1 + a^2] -Sqrt[1 + b^2])], {a, -inf, +inf}, {b, -inf, +inf}, Method -> "LocalAdaptive" ,
IntegrationMonitor :> ((errors = Through[#1@"Error"]) &)];
f[20, 10 ] (* result 1.86704*10^-45*)
Total@errors (* error 1.93674*10^-46*)
Hope it helps!
[1]: https://i.sstatic.net/Oumxp.png