In some sense the point of BigInteger is not so much absolute size as it is unlimited precision. Floating point numbers can be very large too, but have limited precision. BigInteger lets you perform arithmetic with no concern about rounding errors or overflow. The price you pay is that it is hundreds of times slower than arithmetic with ordinary integers or floating point numbers.
As others have pointed out, ulong can hold between 0 to 18,446,744,073,709,551,615, and as long as you stay in that range you can do exact arithmetic. If you go even 1 beyond that range you'll get an overflow, so the answer to your question is use BigInteger if you need exact arithmetic and there is any possibility that any intermediate result will exceed 18,446,744,073,709,551,615.
Most problems in science, engineering and finance can live with the approximations forced by floating point numbers, and can't afford the time cost of BigInteger arithmetic. Most commercial calculations can't live with the approximations of floating point arithmetic, but work within the range 0 to 18,446,744,073,709,551,615, so they can use ordinary arithmetic. BigInteger is needed when using algorithms from number theory which includes things like cryptography (think 50 digit prime numbers). It is also sometimes used in commercial applications when exact calculations are needed, speed is not too important, and setting up a proper fixed decimal point system is too much trouble.
If implemented properly the complexity of BigInteger arithmetic should be a smooth curve. For example the time complexity of multiplication should be O(NM) where N is the number of digits in the first multiplicand, and M is the number of digits in the second multiplicand. Of course there are practical limits in that you could pick N and M so large that the numbers wouldn't fit in your machine.
If you google "Computational complexity of biginteger" you'll get more references than you can shake a stick at. One that speaks directly to your question is this: Comparison of two arbitrary precision arithmetic packages.