python / python/cpython

xmlrpc.client does not properly handle severed HTTPS connections

未关闭
#121,624 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

type-bug
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

Bug description:

I have an xmlrpc.client based script running Python 3.11.2 which has the following setup and problem :

  • it is solliciting an HTTPS endpoint with HTTP/1.1
  • then endpoint is an Apache with a KeepAliveTimeout 5 (seconds)
  • the script makes about 10k XML-RPC requests in a loop for 20 minutes, everything is fine
  • the script works elsewhere for 5min
  • the script then resume to make XML-RPC requests to the same endpoint (same xmlrpc.client object)
  • it fails with this stack trace :
...
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in __call__
    return self.__send(self.__name, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1464, in __request
    response = self.__transport.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1166, in request
    return self.single_request(host, handler, request_body, verbose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1178, in single_request
    http_conn = self.send_request(host, handler, request_body, verbose)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1291, in send_request
    self.send_content(connection, request_body)
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1321, in send_content
    connection.endheaders(request_body)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1076, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.11/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/lib/python3.11/ssl.py", line 1274, in sendall
    v = self.send(byte_view[count:])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1243, in send
    return self._sslobj.write(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2393)

My analysis : the code in xmlrpc.client:request() handles the fact that the TCP connection from a previous HTTP keepalive request might be closed by the server, but only if http.client explicitly raises http.client.RemoteDisconnected. However in my case the remote disconnection was raised as a ssl.SSLEOFError which confused xmlrpc.client and made it fail right away instead of retrying as it is designed to do.

I'm not sure about the fix, but I thought that it would be correct for http.client to assimilate ssl.SSLEOFError to http.client.RemoteDisconnected :

  • semantics seem to match according to the documentation (https://docs.python.org/3/library/ssl.html#ssl.SSLEOFError) : "A subclass of SSLError raised when the SSL connection has been terminated abruptly. Generally, you shouldn’t try to reuse the underlying transport when this error is encountered."
  • it is easier for http.client users to handle, they don't care if the plain socket or the SSL layer reports the remote disconnection

This patch solved the bug described above in my case :

--- /usr/lib/python3.11/http/client.py.orig	2024-05-02 13:59:08.000000000 +0200
+++ /usr/lib/python3.11/http/client.py	2024-07-11 12:22:57.211454581 +0200
@@ -994,10 +994,13 @@
                 self.sock.sendall(datablock)
             return
         sys.audit("http.client.send", self, data)
         try:
             self.sock.sendall(data)
+        except Exception as e:
+            if type(e).__name__ == 'SSLEOFError':
+                raise RemoteDisconnected("SSL layer disconnected")
         except TypeError:
             if isinstance(data, collections.abc.Iterable):
                 for d in data:
                     self.sock.sendall(d)
             else:

It is a bit convoluted because ssl is conditionnaly imported.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先阅读发送路径附近的 Lib/http/client.py,以及请求重试处理附近的 Lib/xmlrpc/client.py。在 Linux 上使用 Python 3.11 重现所描述的 HTTPS keep-alive 超时场景,然后验证突发的 SSL 断开连接会得到预期处理,并且不会立即导致 XML-RPC 请求失败。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。