can someone explain why these 2 operations deliver different results? has it to do with some sort of maximum output? I don't mean the difference in time, but in the calculated result.
l = list(range(100000000)) a = np.arange(100000000) %time np.sum(a ** 2) CPU times: user 132 ms, sys: 217 ms, total: 348 ms Wall time: 347 ms 662921401752298880 %time sum([x ** 2 for x in l]) CPU times: user 23.8 s, sys: 1.32 s, total: 25.1 s Wall time: 25.1 s 333333328333333350000000 As pointed out by pLOPeGG, BLimitless and KaPy3141 numpy's integers can overflow. You can circumvent that by specifying dtype='object' (with minor speedup):
In [1]: import numpy as np In [2]: n = 10_000_000 In [3]: %timeit np.sum(np.arange(n, dtype='object')**2) 2.28 s ± 56.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) In [4]: %timeit sum(i**2 for i in range(n)) 2.6 s ± 534 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # also just for completeness sake # multiplying with itself is considerably faster than squaring explicitly In [5]: %timeit sum(i*i for i in range(n)) 1.29 s ± 242 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) # fastest thing I could come up with: In [6]: %timeit a = np.arange(n, dtype='object'); np.sum(a * a) 939 ms ± 74.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 
numpy still outperform
c methods used for numeric dtypes.
dtype=object it's generally a safe bet to switch to using a list or so@robin has it right: numpy is overflowing the int size. This is a known issue and will hopefully be fixed soon. These links have more info: https://github.com/numpy/numpy/issues/8987 and https://github.com/numpy/numpy/issues/10964
