Speed-up build by avoiding a expensive operation

[abravalheri]

I decided to investigate a little bit the execution time for setuptools' PEP 517 backend after reading some criticism about its speed (it seems that it is much slower than the alternatives) on Python discourse and in other places. I don't really think the difference is relevant, but it would be nice if we can tackle the negative publicity.

So I hacked together this dirty profiling script to see which parts of setuptools take longer to run.

The first time I run it, I saw the following:

(The bright pink shows the function I was selecting to see, in this case exec).

It seems that we are currently paying the price of running exec in most PEP 517 hooks twice... This was very curious, so I came back to a line in the build_meta that I never really understood:

setuptools/setuptools/build_meta.py

Line 174 in 9a60016

exec(compile(code, __file__, 'exec'), locals())

In this line, we are compiling the setup script before executing it, which is curious. I don't know why this is necessary, since exec should also be able to compile the code...

The other part that catch my attention in the profile was dist._install_dependencies (bright pink again):

which seems to be one of the main contributors for the execution time.
When I was looking at the code for this function, I saw that we are trying to find out missing dependencies for entry-points that have extras (which is not very common nowadays since pip simply ignore extras in entry-points).

I decided to quickly hack these two code paths, and I was positively surprised by the huge impact:

(It is clear to me that the precise timings will vary from computer to computer and also depend on which programs the user has open in their machine... but this little experiment seems to be showing a general tendency).

This PR contains these 2 simple changes.
(I don't have a lot of experience with Python profiling, so I might also have implemented something wrong or underestimated something).

Summary of changes

• build_meta: Avoid running compile before exec
  • I honestly don't know if this change has another impact that I am not able to foresee...
• dist: Avoid trying to install dependencies for entry-points that don't have extras.

Closes

Pull Request Checklist

• Changes have tests
• News fragment added in changelog.d/.
  (See documentation for details)

[abravalheri]

The only difference I can think here is that if the setup script has a syntax error, compile would fail so no instruction gets executed. I don't know how exec works, but it could be the case it gets to execute something before failing... I don't think this is relevant though because the PEP 517 hooks are supposed to be executed in isolated subcommands, so it is fine for the hook to crash.

However, I might be failing to see other problems...

Alternatively we could also try.

 try: exec(code, locals()) except Exception: exec(compile(code, __file__, 'exec'), locals())

[jaraco]

It's my understanding that if there is a syntax error, nothing will ever be executed, so no concerns here.

[Joshix-1]

An important difference is that ­— when an exception gets raised — the stacktrace is now broken. The second argument to compile tells python where to find the source code when displaying the stacktrace. That's exactly what #3577 wanted to improve.

Calling compile before exec doesn't seem to have any noticeable effect on performance:

$ python -m timeit -s 'code = "1 + 1 #" + "A"*10000' 'exec(code)' 20000 loops, best of 5: 10.7 usec per loop $ python -m timeit -s 'code = "1 + 1 #" + "A"*10000' 'exec(compile(code, __file__, "exec"))' 20000 loops, best of 5: 10.7 usec per loop 

I don't see how that could make any difference. To execute code the code needs to get compiled before. It doesn't matter if compile gets executed explicitly.

[abravalheri]

@jaraco, is there any chance you could have a look on this?
At first this seems like an easy gain, but I don't know what was the original motivation for the code to be the way it is, so maybe I am failing to see a side-effect...

[abravalheri]

Part of the improvements in this PR was superseded by #3421 so I rebased the proposed changes.

Now the PR only targets the compile + exec issue.

[KOLANICH]

compile calls Py_CompileStringObject

https://github.com/python/cpython/blob/bbcf42449e13c0b62f145cd49d12674ef3d5bf64/Python/bltinmodule.c#L738-L840

exec calls PyRun_StringFlags for the case of strings and PyEval_EvalCodeEx for the case of code.

https://github.com/python/cpython/blob/bbcf42449e13c0b62f145cd49d12674ef3d5bf64/Python/bltinmodule.c#L996-L1111

PyRunStringFlags calls _PyParser_ASTFromString to parse AST and then run_mod, which through the chain of funcs calls PyEval_EvalCode, which looks much simpler than PyEval_EvalCodeEx.

https://github.com/python/cpython/blob/8a0d9a6bb77a72cd8b9ece01b7c1163fff28029a/Python/pythonrun.c#L1587-L1607

run_mod calls _PyAST_Compile (with the optimization level -1, which is the default for compile too, which means retrieving it from _Py_GetConfig()->optimization_level) and then run_eval_code_obj

So the optim8zation flags should be the same.

https://github.com/python/cpython/blob/8a0d9a6bb77a72cd8b9ece01b7c1163fff28029a/Python/pythonrun.c#L1720-L1737

and Py_CompileStringObject calls _PyParserASTFromString and then _PyAST_Compile.