psf / psf/requests

Accessing response.content twice removes forgets read error

Open
#4,965 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug
Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

I had a hard debugging time today because an error in the response stream is only reported when accessing response.content for the first time.

This is especially irritating when running code in a debugger.

Expected Result

If accessing response.content the first time raises an exception I would expect that accessing response.content again would also raise an exception (ideally the same).

Actual Result

Instead after raising on the first get, getting response.content again returns an empty string.

Reproduction Steps

Here is a patch with a new test case for this: error_replay_test.diff.gz.

Basically, it boils down to this:

import requests

response = requests.post("http://connreset.biz/get/incomplete/chunked", stream=True)
try:
    response.content
except Exception:
    # Error handling code, may try something else or fall through
    pass

content = response.content  # empty string

Output of my test case:

$ pipenv run py.test tests/test_lowlevel.py -q --tb=short -k retain
F                                                            [100%]
============================= FAILURES =============================
_______________ test_response_content_retains_error ________________
tests/test_lowlevel.py:343: in test_response_content_retains_error
    assert False, "error response has content: {0!r}".format(content)
E   AssertionError: error response has content: ''
E   assert False
1 failed, 15 deselected in 0.60 seconds

System Information

$ python -m requests.help

Edit: Oops, I used pipenv run python -m requests.help which actually called into system python 2.7. Here comes the real data:

$ pipenv run python3 -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.8+"
  },
  "platform": {
    "release": "4.15.0-43-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.24"
  },
  "using_pyopenssl": false
}

Thanks for looking into this!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/test_lowlevel.py and the named test_response_content_retains_error, then trace how response.content handles a stream read failure. Reproduce the failure with the provided example and confirm that repeated access raises the original error rather than returning an empty string; the focused test should pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.