gh-101607 causes regression in third-party psutil package on macOS #102493
Description
As part of the cPython release process for macOS installers, besides the standard cPython test suite, we run a small set of additional smoke tests. One of these involves an install from source of the psutil package which includes the building of a C extension module. In the run up to the 3.12.0a6 release, we found that our long-unchanged simple test was now failing, a regression from 3.12.0a5 and 3.11.x. The commit causing the failure was bisected to feec49c from GH-101607, a part of issue #101578. From a cursory glance at the traceback and the psutil code, it appears that psutil uses a decorator to translate certain OSError exceptions from its C module into other Python exceptions that its calling code is prepared to handle or ignore. For some reason, that exception translation is now failing. FWIW, this is the first failure of this psutil smoke test in recent memory including previous 3.12.0 alphas. Also, FWIW, this same smoke test does not fail on a reasonably recent Debian Linux system but, of course, that exercises a different platform-dependent code path in psutil.
Because of its dependencies, it is a bit complicated to build and reproduce without changes. Here is one way to do it; it is likely not optimal. This assumes a fairly recent macOS system with Homebrew installed (as described in the devguide) and assumes git checkouts of the cpython and psutil Github repos.
brew install pkg-config openssl@1.1 xz mkdir /tmp/test_psutil cd /tmp/test_psutil git clone https://github.com/giampaolo/psutil.git git clone https://github.com/python/cpython.git # or from an existing repo cd cpython git checkout feec49c40736fc05626a183a8d14c4ebbea5ae28^ # last good commit git clean -fdxq ./configure -q --prefix=/tmp/none --with-pydebug \ --with-openssl="$(brew --prefix openssl@1.1)" make -s -j2 cd .. git -C ./psutil clean -fdxq rm -rf venv ./cpython/python.exe -m venv venv venv/bin/python -m pip install --no-binary psutil ./psutil/ venv/bin/python -c 'import psutil, pprint; pprint.pprint(list(psutil.process_iter()))' The expected result should be something like:
[psutil.Process(pid=0, name='kernel_task', status='running', started='03:25:52'), psutil.Process(pid=1, name='launchd', status='running', started='03:25:52'), psutil.Process(pid=75, name='logd', status='running', started='03:25:54'), psutil.Process(pid=76, name='UserEventAgent', status='running', started='03:25:54'), ... With the failing commit applied (up to and including the current head of the main branch):
git checkout feec49c40736fc05626a183a8d14c4ebbea5ae28 # failing commit
and repeating the above build and test, the result is now something like:
Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/tmp/cpython/Lib/pprint.py", line 55, in pprint printer.pprint(object) File "/private/tmp/cpython/Lib/pprint.py", line 153, in pprint self._format(object, self._stream, 0, 0, {}, 0) File "/private/tmp/cpython/Lib/pprint.py", line 175, in _format rep = self._repr(object, context, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/cpython/Lib/pprint.py", line 455, in _repr repr, readable, recursive = self.format(object, context.copy(), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/cpython/Lib/pprint.py", line 468, in format return self._safe_repr(object, context, maxlevels, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/cpython/Lib/pprint.py", line 619, in _safe_repr orepr, oreadable, orecur = self.format( ^^^^^^^^^^^^ File "/private/tmp/cpython/Lib/pprint.py", line 468, in format return self._safe_repr(object, context, maxlevels, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/cpython/Lib/pprint.py", line 629, in _safe_repr rep = repr(object) ^^^^^^^^^^^^ File "/private/tmp/venv/lib/python3.12/site-packages/psutil/__init__.py", line 389, in __str__ info["name"] = self.name() ^^^^^^^^^^^ File "/private/tmp/venv/lib/python3.12/site-packages/psutil/__init__.py", line 628, in name cmdline = self.cmdline() ^^^^^^^^^^^^^^ File "/private/tmp/venv/lib/python3.12/site-packages/psutil/__init__.py", line 681, in cmdline return self._proc.cmdline() ^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/venv/lib/python3.12/site-packages/psutil/_psosx.py", line 346, in wrapper return fun(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/tmp/venv/lib/python3.12/site-packages/psutil/_psosx.py", line 404, in cmdline return cext.proc_cmdline(self.pid) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [Errno 1] Operation not permitted (originated from sysctl(KERN_PROCARGS2)) Note this seems to come from trying to inspect a privileged process and the point of the exception wrapping is to effectively ignore such errors.
Flagging as a potential release-blocker for @Yhg1s