2

I was expecting a funcdef to bind the closest inner closure to its definition. Apparently it's not the case:

phoo = 4 class Alice: # 'classdef' # <class 'suite'>: phoo = 1 spam = phoo + 11 blah = staticmethod(lambda: phoo + 22) @staticmethod def Blake(): return phoo + 33 

Test:

>>> Alice.spam 12 >>> Alice.blah() 26 >>> Alice.Blake() 37 

It is said that a code block is executed execution frame. In the time when the class definition's 'block' run/executed, spam resolves phoo from inside Alice.

I expected resolution from inside Blake to resolve phoo from Alice. The execution model says,

If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name.

Then it says,

It is said that a code block is executed execution frame.

This decision caused my assumption to go wrong. What is the rationale behind it?

edit: this is python 2 old-style classes; but if noted answers can be on new-style classes. I asked for the reason, however if you could add insider technical explanation, it is very welcome too!

12
  • possible duplicate of The scope of names defined in class block doesn't extend to the methods' blocks. Why is that? Commented Apr 17, 2015 at 14:16
  • 1
    phoo, spam and blah are not "free" variables like it looks like you expect, rather they're bound as attributes to the class Alice. Even inside your Blake function, you'll have to use attribute access (Alice.spam) to get them (or if Blake were a method, self.spam). Commented Apr 17, 2015 at 14:19
  • Are you asking the technical way that this works under the covers, or the reason why it was designed that way? (Of course those aren't entirely unrelated, but they're definitely not the same question.) Commented Apr 17, 2015 at 14:21
  • Also, is this Python 2 or 3? And, if it's 2, are you intentionally asking about old-style classes? Commented Apr 17, 2015 at 14:24
  • 1
    Well, you're probably right about that… but the details behind the decisions made in the old days are mostly lost to history, except when Guido happens to remember something interesting and write a Python history blog post… Commented Apr 17, 2015 at 14:36

1 Answer 1

3

From an intuitive point of view, the answer is pretty simple:

Free variables in a function definition capture variables in the enclosing scope. But class attributes aren't variables, they're class attributes; you have to access them as Alice.spam or self.spam, not as spam. Therefore, spam doesn't capture the outer spam because there is no outer spam.


But under the covers, this isn't really true.

For a new-style class, while the class definition's body is being executed, spam actually is a local variable in the scope of that body; it's only when the metaclass (type, in this case) is executed that the class attributes are created from those locals.[1]

For an old-style class, it's not completely defined what happens, so you pretty much have to turn to the implementation. In particular, there's no step where the metaclass is executed with the class definition's locals to generate the class object. But for the most part, it works pretty much as if that were the case.


So, why doesn't spam bind to that local?

A free variable can only bind to a closure cell from an outer scope, which is a special kind of local variable. And the compiler only creates a closure cell for a variable in a function definition when a local function accesses it. It doesn't create closure cells for variables in class definitions.


So if spam doesn't bind to Alice.spam, what does it bind to? Well, by the usual LEGB rules, if there's no local assignment, and no enclosing cell variable, it's a global.


Some of the above may be hard to understand without an example, so:

>>> def f(): ... a=1 ... b=2 ... def g(): ... b ... return g >>> f.__code__.co_cellvars # cell locals, captured by closures ('b',) >>> f.__code__.co_varnames # normal locals ('a', 'g') >>> g = f() >>> g.__code__.co_freevars # free variables that captured cells ('b',) >>> class Alice: ... a=1 ... b=2 ... def f(): ... b >>> Alice.f.__func__.__code__.co_freevars () >>> Alice.f.__func__.__code__.co_varnames () >>> Alice.f.__func__.__code__.co_names # loosely, globals ('b',) 

If you're wondering where co_cellvars and the like are specified… well, they're not, but the inspect module docs give a brief summary of what they mean.


If you understand CPython bytecode, it's also worth calling dis on all of these chunks of code to see the instructions used for loading and saving all these variables.


So, the big question is, why doesn't Python generate cells for class definitions?

Unless Guido remembers, and finds it interesting enough to write a Python history blog post about this, I'm not sure we'll ever know the answer. (You could, of course, try asking him—a comment on his blog or an email to whichever mailing list seems most relevant is probably the best way.)

But here's my guess:

Cells are implemented as indices into an array stored in a code object. When the function is called, its frame gets an matching array of objects. When a local function definition is executed inside that function call, the free variables are bound to references to the cell slots in the frame.

Classes don't have __code__ members (or, pre-2.6, co_code). Why? Because a class definition is executed as soon as it's defined, and never executed again, so why bother? This means there's nowhere to stash a cell, and nothing for it to reference. On top of that, the execution frame always goes away as soon as the execution finishes, because there can't be any external references to it.

Of course you could change that: add __code__ members to classes, create cells in them, and then, if someone closed over those cells, that would keep the frame alive after execution just as it does with functions. Would that be a good idea? I don't know. My guess is that nobody asked the question when Python classes were first being defined. While it's obvious now how much class definitions and function definitions have in common, I think that's an instance of Guido's time machine—he made a design decision without realizing that it would turn out to solve problems nobody would raise until a decade later.


[1] Some of these details may be CPython-specific. For example, I believe it's technically legal for an implementation to make a closure cell for every local in a function, or to use some other mechanism that's equivalent to that. For example, if you do exec('spam=3') in an inner function, all the language reference says is that it's not guaranteed that it will affect the outer function's spam, not that it's guaranteed not to do so.

Sign up to request clarification or add additional context in comments.

3 Comments

off: is LISP a good toy for better understanding free variables from a historical perspective?
@naxa: Yes and no. On the one hand, Lisp free variables are definitely simpler (there's only one kind of scope, and, in Python terms, every local is a cell variable), and they were the inspiration for everything that came later. On the other hand, early versions of Lisp did dynamic scope rather than lexical, which is completely different. But on the third hand, the reasons why Lisp switched from dynamic scope to lexical are well documented and discussed, which is very handy.
@naxa: Also, of course, earlier Lisps didn't have classes, and Common Lisp's classes are built out of closures, which kind of changes the whole nature of a question like this one…

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.