Unclosed socket in urllib when ftp request times out after connect #140691
Description
Bug report
Bug description:
Spotted in the log of "Log from "buildbot/ARM64 macOS 3.x"; this doesn't fix the ftp test but when the operation times out in the test CPython should close the socket
This seems to be originating here:
Lines 1539 to 1556 in f5394c2
In particular, fw = self.connect_ftp succeeds which means fw contains an open connection but fp, retrlen = fw.retrfile(file, type) fails and the except doesn't result in a fw.close().
Log from "buildbot/ARM64 macOS 3.x"
Exception ignored while finalizing socket <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)>: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile conn, retrlen = self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd conn = socket.create_connection((host, port), self.timeout, source_address=self.source_address) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection raise exceptions[0] File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection sock.connect(sa) ~~~~~~~~~~~~^^^^ TimeoutError: [Errno 60] Operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice return func(*args, **kwargs) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen return opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open response = self._open(req, data) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open result = self._call_chain(self.handle_open, protocol, protocol + '_open', req) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain result = func(*args) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open raise URLError(f"ftp error: {exp}") from exp urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice last_exc = e ^^^^^^^^ ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)> Warning -- Unraisable exception Exception ignored while finalizing socket <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile conn, retrlen = self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd conn = socket.create_connection((host, port), self.timeout, source_address=self.source_address) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection raise exceptions[0] File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection sock.connect(sa) ~~~~~~~~~~~~^^^^ TimeoutError: [Errno 60] Operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice return func(*args, **kwargs) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen return opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open response = self._open(req, data) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open result = self._call_chain(self.handle_open, protocol, protocol + '_open', req) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain result = func(*args) File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open raise URLError(f"ftp error: {exp}") from exp urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice last_exc = e ^^^^^^^^ ResourceWarning: unclosed <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-140691: Close FTP connection on error in urllib.request #140694
- gh-140691: urllib.request: Close FTP control socket if data socket can't connect #140835
- [3.14] gh-140691: urllib.request: Close FTP control socket if data socket can't connect (GH-140835) #141555
- [3.13] gh-140691: urllib.request: Close FTP control socket if data socket can't connect (GH-140835) #141657