Error in AST recursion depth tracking change of gh-95185 #106905
Description
Bug report
The change in 0047447 seems to miss the recusrion depth adjustment in case of an error. As an example for some of the generated code:
PyObject* ast2obj_mod(struct ast_state *state, void* _o) { mod_ty o = (mod_ty)_o; PyObject *result = NULL, *value = NULL; PyTypeObject *tp; if (!o) { Py_RETURN_NONE; } if (++state->recursion_depth > state->recursion_limit) { PyErr_SetString(PyExc_RecursionError, "maximum recursion depth exceeded during ast construction"); return 0; } switch (o->kind) { case Module_kind: tp = (PyTypeObject *)state->Module_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; value = ast2obj_list(state, (asdl_seq*)o->v.Module.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; ... } state->recursion_depth--; return result; failed: Py_XDECREF(value); Py_XDECREF(result); return NULL; } Note that the failed code path is missing the state->recursion_depth--; statement.
I found this as I'm trying to track down where spurious SystemError: AST constructor recursion depth mismatch errors in Python 3.11 are coming from. E.g.
File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 358, in unpack_ndarray_from_bytes return unpack_ndarray(source) ^^^^^^^^^^^^^^^^^ File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 305, in unpack_ndarray sink = PlainNumpySink(source.metadata) ^^^^^^^^^^^^^^^^^ File "/env/lib/python3.11/site-packages/bloscpack/numpy_io.py", line 136, in __init__ dtype_ = ast.literal_eval(metadata['dtype']) ^^^^^^^^^^^^^^^^^ File "/env/lib/python3.11/ast.py", line 64, in literal_eval node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval') ^^^^^^^^^^^^^^^^^ File "/env/lib/python3.11/ast.py", line 50, in parse return compile(source, filename, mode, flags, ^^^^^^^^^^^^^^^^^ SystemError: AST constructor recursion depth mismatch (before=48, after=47) Your environment
Reproduced in Python 3.11 but the code in main looks the same.
Linked PRs
- gh-106905: avoid incorrect SystemError about recursion depth mismatch #106906
- [3.12] gh-106905: avoid incorrect SystemError about recursion depth mismatch (GH-106906) #112032
- [3.11] gh-106905: Avoid incorrect SystemError about recursion depth mismatch (#106906) #112849
- gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. #113035
- [3.12] gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) #113472
- [3.11] gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) (GH-113472) #113476