3

I am trying to optimise some python code, via testing (timing) various functions using timeit.

I have found that I am getting different speeds depending on whether a variable is a keyword argument or within the function.

That is:

def test_function(A = value()): #rest of function....

Is returning a different result than:

def test_function(): A = value() #rest of function ...

I would have figured they would have very similar results - I am guessing I am not understanding / missing something here...

(doing a 10,000 loops for the tests too)

Improve this question
3
  • make sure you understand the moral of doing value(). If this is some random id generator, you will be in trouble. That one id will be forever default. So make sure your application is okay by calling value(). Commented Nov 14, 2012 at 7:16
  • There's a fast_function speed hack in CPython, for a positional-only call if the function object lacks default arguments or cellvars/freevars setup. Otherwise it uses PyEval_EvalCodeEx. My tests with 255 arguments show about a 5-7% performance boost for no defaults vs defaults. But more importantly, using 255 keyword arguments was about 12-13 times slower in either case, since it's so much more work to setup and evaluate a call with keywords. Commented Nov 14, 2012 at 10:34
  • That said, evaluating the code of most functions will completely dwarf the call overhead. Commented Nov 14, 2012 at 10:40

3 Answers 3

Reset to default
11

Keyword arguments are evaluated once at function definition time. So in your first example value() is called exactly once, no matter how often you call the test function. If value() is expensive-ish this explains the difference in runtime between the two versions.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Right, thanks - that is the something I was missing! Much appreciated.
4

There's no reason your two functions should be expected to do the same thing, let alone have the same performance characteristics.

def test_function(A = value()): #rest of function....

This function doesn't have a "keyword argument"; there's no such thing. It has an argument with a default value. Any parameter for any function (some recalcitrant built-ins aside) can be passed by keyword or by position, but arguments are not intrinsically "keyword arguments".

The only association between keyword arguments and default values is that when you have multiple arguments with default values, the only way to supply an explicit value to a later argument while accepting the default for an earlier one is to pass the later argument by keyword.

The huge difference between the two functions is that when you declare a default value for A, it's a default value, not some code that will regenerate the value each time if an explicit value isn't provided. When you say this:

def test_function(A = value()): #rest of function....

You're setting a default value for A. As in any other context, when you provide a complex expression where Python needs a value, Python will evaluate that expression and then use the resulting value. So when you set the default value for A, at function definition time, it gets set to whatever value() returns at that time. Then that one single value is the default value for A.

def test_function(): A = value() #rest of function ...

In this function, value() is evaluated every time the function is called. So if value() is expensive, then this version will take much longer than the first version. But if value() returns an object which you later mutate, then the default-argument version will be always using the one single object, in whatever state it was in at the time the function was called, while the second version will be constructing a new value every time. Which version you use should be determined by the semantics you want your program to have.

Improve this answer

3 Comments

Thanks for your considered response! I had no-idea. Cheers.
There most certainly is such a thing as a keyword argument, even in Python 2, and in Python 3 there are keyword-only arguments.
@agf That tutorial you linked to discusses keyword arguments exactly as I view them. A function just has formal parameters (and an optional **kwargs, so I suppose you could argue that any particular names it listens to from **kwargs are "keyword parameters"). At call time arguments may be passed by position or by keyword; it's not a property of the function's formal parameter. It's true that I omitted mentioning **kwargs (or keyword only parameters in Python3) to simplify the message. The core point I was making is "keyword" has nothing to do with "default value".
4

There are discussions as to why this method isn't the best to determine efficacy of approaches, but if you use dis to inspect the bytecode of the functions, you can see that they are structured in different ways, namely that t1 evaluates its default argument at the time it is defined, and therefore does not require it to be redefined on subsequent function calls:

>>> import dis >>> def t1(A=1): ... pass >>> def t2(): .... A=1 >>> dis.dis(t1) 2 0 LOAD_CONST 0 (None) 3 RETURN_VALUE >>> dis.dis(t2) 2 0 LOAD_CONST 1 (1) 3 STORE_FAST 0 (A) 6 LOAD_CONST 0 (None) 9 RETURN_VALUE
Improve this answer

2 Comments

Cheers - haven't come across the dis module before - looks very useful,
@djmac Definitely can come in handy sometimes. Good question by the way!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.