psf / psf/requests

requests.exceptions.ReadTimeout is not raised when response is chunked

Open
#2,392 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Breaking API Change Bug Planned
Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

https://gist.github.com/nickjoyce-wf/c536f2573ef124115f86

When a request has a timeout param supplied and the response is chunked but does not return enough data in time, requests.exceptions.ReadTimeout should be raised.

$ mkvirtualenv test
(test)$ pip install requests werkzeug
(test)$ python server.py

In another terminal:

$ workon test
(test)$ python client.py

Expected results:

requests.exceptions.ReadTimeout is raised due to response not being sent within the timeout specified by requests.get.

Actual results:

Traceback (most recent call last):
  File "client.py", line 3, in <module>
    requests.get('http://127.0.0.1:8000/', timeout=0.5)
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/api.py", line 65, in get
    return request('get', url, **kwargs)
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/api.py", line 49, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/sessions.py", line 610, in send
    r.content
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/models.py", line 730, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/Users/nick/.virtualenvs/test/lib/python2.7/site-packages/requests/models.py", line 662, in generate
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out.

Recorded http request/response via Wireshark:

GET / HTTP/1.1
Host: 127.0.0.1:8000
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.5.1 CPython/2.7.9 Darwin/13.4.0

HTTP/1.0 200 OK
Content-type: text/plain
Transfer-Encoding: chunked
Connection: close
Server: Werkzeug/0.9.6 Python/2.7.9
Date: Fri, 26 Dec 2014 17:51:21 GMT

Tested with requests==2.5.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with requests/sessions.py and requests/models.py, following the reported path through response content and iter_content; reproduce the chunked-response case using the linked server.py and client.py setup. Done means the timeout scenario raises requests.exceptions.ReadTimeout rather than requests.exceptions.ConnectionError, with a regression test covering the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.