8

I saw a comment that lead me to the question Why does Python code run faster in a function?.

I got to thinking, and figured I would try it myself using the timeit library, however I got very different results:

(note: 10**8 was changed to 10**7 to make things a little bit speedier to time)

>>> from timeit import repeat >>> setup = """ def main(): for i in xrange(10**7): pass """ >>> stmt = """ for i in xrange(10**7): pass """ >>> min(repeat('main()', setup, repeat=7, number=10)) 1.4399558753975725 >>> min(repeat(stmt, repeat=7, number=10)) 1.4410973942722194 >>> 1.4410973942722194 / 1.4399558753975725 1.000792745732109
  • Did I use timeit correctly?
  • Why are these results less 0.1% different from each other, while the results from the other question were nearly 250% different?
  • Does it only make a difference when using CPython compiled versions of Python (like Cython)?
  • Ultimately: is Python code really faster in a function, or does it just depend on how you time it?
Improve this question
5
  • 1
    I think this is largely dependent on the implementation (so both version and distribution (i.e. normal Python vs CPython etc.) are important). Also, you should try running the exact same code as in that other question (and timing it the same way), just for a proper comparison. Commented Mar 8, 2013 at 5:47
  • 1
    This post is insightful. Commented Mar 8, 2013 at 5:53
  • So, it sounds like it matters in compiled versions of Python (CPython, PyPy, etc.), but in plain old vanilla Python it doesn't make a lick of difference! Commented Mar 8, 2013 at 6:04
  • 2
    @WesleyBaugh: Plain old vanilla Python is CPython. Commented Mar 8, 2013 at 7:09
  • @DietrichEpp That's true! I was actually thinking of Cython (though, now I'm not sure if that makes much of a difference in this case). Commented Mar 8, 2013 at 7:13

2 Answers 2

Reset to default
11

The flaw in your test is the way timeit compiles the code of your stmt. It's actually compiled within the following template:

template = """ def inner(_it, _timer): %(setup)s _t0 = _timer() for _i in _it: %(stmt)s _t1 = _timer() return _t1 - _t0 """

Thus stmt is actually running in a function, using the fastlocals array (i.e. STORE_FAST).

Here's a test with your function in the question as f_opt versus the unoptimized compiled stmt executed in the function f_no_opt:

>>> code = compile(stmt, '<string>', 'exec') >>> f_no_opt = types.FunctionType(code, globals()) >>> t_no_opt = min(timeit.repeat(f_no_opt, repeat=10, number=10)) >>> t_opt = min(timeit.repeat(f_opt, repeat=10, number=10)) >>> t_opt / t_no_opt 0.4931101445632647
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

This is great, and shows an advanced understanding of the inner workings of Python. In your experience, does this influence how we should code (during an optimization stage)?
Good design already calls for using functions and minimizing your use of global variables, but definitely avoid accessing global variables in tight loops that need speed. You can find more advice in the wiki: PythonSpeed.
1

It comes down to compiler optimization algorithms. When performing Just-in-time compilation, it is much easier to identify frequently used chunks of code if they're found in functions.

The efficiency gains really would depend on the nature of the tasks being performed. In the example you gave, you aren't really doing anything computationally intensive, leaving fewer opportunities to achieve gains in efficiency through optimization.

As others have pointed out, however, CPython does not do just-in-time compilation. When code is compiled, however, C compilers will often execute them faster.

Check out this document on the GCC compiler: http://gcc.gnu.org/onlinedocs/gcc/Inline.html

Improve this answer

7 Comments

It sounds like this is true for CPython (since it is compiled), but not for vanilla Python since it is not.
@WesleyBaugh: Wait... isn't CPython the same as "vanilla" Python? Or is "vanilla" python something else?
CPython barely does any optimization. The extent of optimization is eliminating some useless loads and stores. CPython has no JIT, and it simply executes bytecode sequentially as an interpreter. (The same is not true for many other implementations, e.g. PyPy, IronPython or Jython, which all have JITs backing them).
(-1: this answer is too general to be useful in a Python-specific question, and is in fact wrong in the context of the most popular Python inteprerter.)
@DietrichEpp Thanks for the correction. I think I was thinking of Cython.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.