It looks like the warning registry does not get flushed properly: ./python -m test test___all__ test_binhex 0:00:00 load avg: 1.55 Run tests sequentially 0:00:00 load avg: 1.55 [1/2] test___all__ 0:00:01 load avg: 1.55 [2/2] test_binhex test test_binhex crashed -- Traceback (most recent call last): File "/Users/remi/src/cpython/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner refleak = _runtest_inner2(ns, test_name) File "/Users/remi/src/cpython/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2 the_module = importlib.import_module(abstest) File "/Users/remi/src/cpython/Lib/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 790, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/Users/remi/src/cpython/Lib/test/test_binhex.py", line 10, in <module> import binhex File "/Users/remi/src/cpython/Lib/contextlib.py", line 124, in __exit__ next(self.gen) File "/Users/remi/src/cpython/Lib/test/support/__init__.py", line 1166, in _filterwarnings raise AssertionError("filter (%r, %s) did not catch any warning" % AssertionError: filter ('', DeprecationWarning) did not catch any warning test_binhex failed == Tests result: FAILURE == 1 test OK. 1 test failed: test_binhex Total duration: 2.0 sec Tests result: FAILURE It's not a very issue but does appear when running refleaks on the whole stdlib.