C (gcc), 87 76 74 72 bytes
Optimization of innat3's C# (Visual C# Interactive Compiler), 100 bytes
f(n,i,t,r,m){for(t=0,m=n;r-2;t++)for(r=i=1,n+=n<m?t:-t;i<n;n%++i||r++);} Optimization of innat3's C# (Visual C# Interactive Compiler), 100 bytes
f(n,i,t,r,m){for(t=0,m=n;r-2;t++)for(r=i=1,n+=n<m?t:-t;i<n;n%++i||r++);}