python / python/cpython

xmlrpc.client does not properly handle severed HTTPS connections

未關閉
#121,624 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

I have an xmlrpc.client based script running Python 3.11.2 which has the following setup and problem :

  • it is solliciting an HTTPS endpoint with HTTP/1.1
  • then endpoint is an Apache with a KeepAliveTimeout 5 (seconds)
  • the script makes about 10k XML-RPC requests in a loop for 20 minutes, everything is fine
  • the script works elsewhere for 5min
  • the script then resume to make XML-RPC requests to the same endpoint (same xmlrpc.client object)
  • it fails with this stack trace :
...
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in __call__
    return self.__send(self.__name, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1464, in __request
    response = self.__transport.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1166, in request
    return self.single_request(host, handler, request_body, verbose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1178, in single_request
    http_conn = self.send_request(host, handler, request_body, verbose)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1291, in send_request
    self.send_content(connection, request_body)
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1321, in send_content
    connection.endheaders(request_body)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1076, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.11/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/lib/python3.11/ssl.py", line 1274, in sendall
    v = self.send(byte_view[count:])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1243, in send
    return self._sslobj.write(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2393)

My analysis : the code in xmlrpc.client:request() handles the fact that the TCP connection from a previous HTTP keepalive request might be closed by the server, but only if http.client explicitly raises http.client.RemoteDisconnected. However in my case the remote disconnection was raised as a ssl.SSLEOFError which confused xmlrpc.client and made it fail right away instead of retrying as it is designed to do.

I'm not sure about the fix, but I thought that it would be correct for http.client to assimilate ssl.SSLEOFError to http.client.RemoteDisconnected :

  • semantics seem to match according to the documentation (https://docs.python.org/3/library/ssl.html#ssl.SSLEOFError) : "A subclass of SSLError raised when the SSL connection has been terminated abruptly. Generally, you shouldn’t try to reuse the underlying transport when this error is encountered."
  • it is easier for http.client users to handle, they don't care if the plain socket or the SSL layer reports the remote disconnection

This patch solved the bug described above in my case :

--- /usr/lib/python3.11/http/client.py.orig	2024-05-02 13:59:08.000000000 +0200
+++ /usr/lib/python3.11/http/client.py	2024-07-11 12:22:57.211454581 +0200
@@ -994,10 +994,13 @@
                 self.sock.sendall(datablock)
             return
         sys.audit("http.client.send", self, data)
         try:
             self.sock.sendall(data)
+        except Exception as e:
+            if type(e).__name__ == 'SSLEOFError':
+                raise RemoteDisconnected("SSL layer disconnected")
         except TypeError:
             if isinstance(data, collections.abc.Iterable):
                 for d in data:
                     self.sock.sendall(d)
             else:

It is a bit convoluted because ssl is conditionnaly imported.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

先閱讀傳送路徑附近的 Lib/http/client.py,以及請求重試處理附近的 Lib/xmlrpc/client.py。在 Linux 上使用 Python 3.11 重現所描述的 HTTPS keep-alive 逾時情境,接著確認突發的 SSL 中斷連線會得到預期的處理,且不會立即導致 XML-RPC 請求失敗。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
networking
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。