python / python/cpython

Unclosed socket in urllib when ftp request times out after connect

未關閉
#140,691 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib topic-IO type-bug
主要語言
Python
星號
77.2k
分支
35.9k
PR 指標待擷取

描述

Bug report

Bug description:

Spotted in the log of "Log from "buildbot/ARM64 macOS 3.x"; this doesn't fix the ftp test but when the operation times out in the test CPython should close the socket

This seems to be originating here:
https://github.com/python/cpython/blob/f5394c257ce4118394fe5ff561a2b0ad238a8d65/Lib/urllib/request.py#L1539-L1556

In particular, fw = self.connect_ftp succeeds which means fw contains an open connection but fp, retrlen = fw.retrfile(file, type) fails and the except doesn't result in a fw.close().

Log from "buildbot/ARM64 macOS 3.x" 

Exception ignored while finalizing socket <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
                                    source_address=self.source_address)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection
    raise exceptions[0]
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
TimeoutError: [Errno 60] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
    last_exc = e
    ^^^^^^^^
ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
                                    source_address=self.source_address)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection
    raise exceptions[0]
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
TimeoutError: [Errno 60] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
    last_exc = e
    ^^^^^^^^
ResourceWarning: unclosed <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs
  • gh-140694
  • gh-140835
  • gh-141555
  • gh-141657

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

研究方向

閱讀 Lib/urllib/request.py 中的 ftp_open，尤其是 connect_ftp 以及報告中描述的 retrfile 例外路徑。檢查 test/test_urllib2net.py，並驗證 timeout 情況不再留下未關閉的 socket 或發出 ResourceWarning。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
networking
Issue 類型
缺陷
難度
2/5
預估耗時
1-3 小時
活躍度
停滯
描述清晰度
描述清楚
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。