`connect` timeout is *not* a TCP connect timeout, but a TCP connect + send the whole HTTP request timeout
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
connect timeout is not a TCP connect timeout, but a TCP connect + send the whole HTTP request timeout.
Expected Result
I expect that the documentation is correct about connect timeout:
The advanced documentation says:
The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the connect()) call on the socket. It’s a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.
Actual Result
The connect timeout seems to be the TCP connect timeout + sending the whole HTTP request.
Reproduction Steps
dd if=/dev/zero of=zero.100MB bs=100M count=1
import requests
files = {'file': open('zero.100MB', 'rb')}
requests.post('somehost', timeout=(1, 1000), files=files)
(It needs a host accepting the large POST, I couldn't find any public right now...)
Result:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 936, in _send_output
self.send(message_body)
File "/usr/lib/python3.5/http/client.py", line 908, in send
self.sock.sendall(data)
File "/usr/lib/python3.5/ssl.py", line 891, in sendall
v = self.send(data[count:])
File "/usr/lib/python3.5/ssl.py", line 861, in send
return self._sslobj.write(data)
File "/usr/lib/python3.5/ssl.py", line 586, in write
return self._sslobj.write(data)
socket.timeout: The write operation timed out
System Information
$ python -m requests.help
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "2.8"
},
"implementation": {
"name": "CPython",
"version": "3.6.8"
},
"platform": {
"release": "4.15.0-64-generic",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.22.0"
},
"system_ssl": {
"version": "1010100f"
},
"urllib3": {
"version": "1.25.7"
},
"using_pyopenssl": false
}
Analysis
Digging into urllib3:
https://github.com/urllib3/urllib3/blob/37ba61a8b8120cbd866d057eaa3936f4b140dee0/src/urllib3/connectionpool.py#L370-L390
It sets the connect timeout on the connection, then validate https, then sends the full request, and only then sets the read timeout on the connection socket.
That being said I could not find any urllib3 doc explicitly defining the connect timeout as requests does...
Finally:
https://docs.python.org/3.8/library/socket.html#socket.socket.sendall
Changed in version 3.5: The socket timeout is no more reset each time data is sent successfully. The socket timeout is now the maximum total duration to send all data.
So maybe it did work as a TCP connect timeout + write idle timeout before python 3.5 (which is still not just TCP connect timeout, but closer to it), but now it is clearly not.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the advanced timeout documentation and the referenced urllib3 connectionpool.py section around lines 370-390, then compare the behavior with Python's socket.sendall documentation. Reproduce the large POST timeout case using the provided commands and determine whether the documented semantics or the timeout implementation should change. Done means the connect timeout behavior and documentation agree.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100