bpo-45753: Move function object struct to an internal header.

[markshannon]

The layout of the function object struct should not be exposed in the API. It is not part of the ABI.

We will want to change the layout to improve performance of Python-to-Python calls and other purposes, and having it in a public header prevents us from doing that.

@encukou does this seem OK to you?

https://bugs.python.org/issue45753

[encukou]

Not part of the stable ABI, but since it was exposed in the C-API, I expect that people are using it. The backwards compatibility policy applies.
If you get an exception from the policy, please add some instructions to "Porting to Python 3.11": https://docs.python.org/3.11/whatsnew/3.11.html#id2

[markshannon]

The struct is exposed in a header file, but that doesn't make it part of the C API.

I've said it before, and I'll say it again: the fields of a struct cannot be part of an API, unless that struct is used as an input to an API function, and even in that case, the meaning of those fields must be documented.

Fishing fields out of a struct has no meaning unless the meaning of those fields is documented.

[encukou]

The struct is exposed in a header file, but that doesn't make it part of the C API.

What is the C API, then?
You know my opinion – the C API is, unfortnately, historically defined by the source/headers.
Your opinion is also consistent.

[markshannon]

What is the C API, then? You know my opinion – the C API is, unfortnately, historically defined by the source/headers. Your opinion is also consistent.

The problem is that source/headers cannot define an API by themselves. An API is not just syntax, it is semantics as well.

Why is this not the definition of the C API? https://docs.python.org/3/c-api/index.html

[encukou]

An API is not just syntax, it is semantics as well.

Right, so you'd also look how CPython uses the API. AFAIK, that's historically how you'd use the underdocumented parts of the API.

As for the "Porting to Python 3.11" section in What's new, I think we should write for a target audience of someone who's inherited an old dusty C extension, and needs to port it without knowing much about the C-API (and especially the undocumented parts). I suggest writing something for any change of API that's been around for a decade – even if it's private.

[markshannon]

Ok, this is more controversial than I had hoped.
Maybe I'll bundle up moving the function, frame and codeobject headers into a single PEP.

[encukou]

Thanks. Getting other opinions sounds like the right thing.

[vstinner]

Maybe I'll bundle up moving the function, frame and codeobject headers into a single PEP.

A PEP to make PyThreadState and PyFrameObject opaque sounds like a good idea. I expect less users of PyFunctionObject, but I didn't look at this structure yet. I'm working on making these structures for 2 years :-)

• PyThreadState: https://bugs.python.org/issue39947
• PyFrameObject: https://bugs.python.org/issue40421

[vstinner]

IMO a PEP is a good way to ask for an exception for the PEP 387 policy ;-) So we can consider to implement such PEP in Python 3.11.