generator is slower compared to list comprehension

Question

From what I know, we may use generator when we want to use the values only once. Using the 2 examples below, my logic is that the 2nd one should be faster, because the first one creates a list first and then loop over the values.. while the 2nd only process the values from the generator. Yet, when I calculate the time, the list comprehension is always faster then the generator. Why is this?

1st:

x = [] a = time.perf_counter() for j in [i**2 for i in range(20000)]: x.append(j) print( time.perf_counter() - a )

2nd:

x = [] a = time.perf_counter() for j in (i**2 for i in range(20000)): x.append(j) print( time.perf_counter() - a )

Which python version are you using? Because range returned a list in python 2 and is returning a generator in python 3 ... — quant
– quant, Commented Nov 16, 2018 at 11:28
Possible duplicate of List comprehension vs generator expression's weird timeit results? — jpp
– jpp, Commented Nov 16, 2018 at 12:10

AKX · Accepted Answer · 2018-11-16 11:57:10Z

Yeah, generators and genexprs are generally (heh) slower than list comprehensions, but on the other hand they're lazily evaluated, and you don't have to pay the memory cost for a fully precomputed list either. I imagine the speed difference is caused by call frame overhead with the (implicit or explicit) yield throwing values around.

Using your code, but timeit to measure it, and with a third version using a generator function:

import timeit def f1(): x = [] for j in [i**2 for i in range(20000)]: x.append(j) return x def f2(): x = [] for j in (i**2 for i in range(20000)): x.append(j) return x def f3(): def gen(): for i in range(20000): yield i ** 2 x = [] for j in gen(): x.append(j) return x print(timeit.timeit(f1, number=100)) print(timeit.timeit(f2, number=100)) print(timeit.timeit(f3, number=100))

The results (Python 3.7.0) seem to point to genexprs being exactly as fast as generator functions, about 4-5% slower than the list comprehension.

f1 = 2.882695159 f2 = 3.0303254170000002 f3 = 3.002670741

Thanks. So using the generator is more efficient in memory, but that does not always mean it is faster than the ones that cost more memory?

Collectives™ on Stack Overflow

generator is slower compared to list comprehension

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related