`BaseHTTPRequestHandler` hides `TimeoutError` from `handle_error()`

BaseHTTPRequestHandler と issue に記載されたリクエスト処理経路から始め、続いて提供されたテストケースを再現して、どの TimeoutError 例外が handle_error() に到達するかを確認します。ハンドラーが発生させたタイムアウトが handle_error() を通じて報告され、既存のリクエストソケットのタイムアウト動作が失われなければ完了です。

Bug report

If anything in your method handlers (e.g. do_GET()) ends up throwing a TimeoutError, then that unhandled exception is never forwarded to handle_error(), and hence you do not get a chance to do whatever reporting you have set up there.

The cause of this is an undocumented timeout handling that was added in bpo-6267. It blindly assumes that any TimeoutError must come from the request socket and that it was requested and hence expected. That is not the case for any mildly complex request handler that might communicate with other services.

The timeline for this issue is:

?.? to 2.6: No exceptions are hidden from handle_error()
2.7 to 3.2: recv() and send() timeouts are hidden. Other timeouts, e.g. connect(), are not.
3.3 to 3.9: Same situation, but by pure luck since socket.timeout was not converted to TimeoutError yet
3.10: All timeouts are hidden, not just for recv() and send()

Your environment

• CPython versions tested on: 2.7.18, 3.2.3, 3.5.10, 3.10.7
• Operating system and architecture: Debian 7 x86_64 and Fedora 35 x86_64

Test case

This test case should give the output:

Got error!
Got error!

But instead gives:

127.0.0.1 - - [25/Nov/2022 13:48:49] Request timed out: TimeoutError(110, 'Connection timed out')
127.0.0.1 - - [25/Nov/2022 13:48:50] Request timed out: TimeoutError('timed out')

For 3.9 or older you get a mix.

Code: (change simulate if you want to have the actual socket code throw the exceptions)

#!/usr/bin/python3

import os
import errno
import socket
import threading
from time import sleep
try:
	from http.server import HTTPServer, BaseHTTPRequestHandler
except ImportError:
	from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler

simulate = True

class ConnectRequestHandler(BaseHTTPRequestHandler):
	def do_GET(self):
		if simulate:
			raise socket.error(errno.ETIMEDOUT, "Timeout")
		else:
			socket.create_connection(("1.2.3.4", 9000))

class RecvRequestHandler(BaseHTTPRequestHandler):
	def do_GET(self):
		if simulate:
			raise socket.timeout(errno.ETIMEDOUT, "Timeout")
		else:
			(sock, dummy) = socket.socketpair()
			sock.settimeout(1)
			while sock.recv(1024):
				pass

class Server(HTTPServer):
	def handle_error(self, request, client_address):
		print("Got error!")

def run_connect_server():
	s = Server(("127.0.0.1", 8998),
	           ConnectRequestHandler)
	s.handle_request()
	
def run_recv_server():
	s = Server(("127.0.0.1", 8999),
	           RecvRequestHandler)
	s.handle_request()

threading.Thread(target=run_connect_server).start()
threading.Thread(target=run_recv_server).start()
sleep(0.5)

s = socket.create_connection(("127.0.0.1", 8998))
s.send(b'GET / HTTP/1.0\r\n\r\n')
while s.recv(1024):
	pass
s.close()

s = socket.create_connection(("127.0.0.1", 8999))
s.send(b'GET / HTTP/1.0\r\n\r\n')
while s.recv(1024):
	pass
s.close()

sleep(0.5)

Linked PRs

• gh-99806