Add support for setting resp.stream to a requests.Response or urllib3.HTTPResponse object
- Dominant language
- Python
- Stars
- 9.8k
- Forks
- 1k
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 7
Description
We already have support for streaming via a file-like project, but it is also common to fetch upstream resources via the requests library, and so it would be really helpful if Falcon supported this natively. Internally, the object can be tested to see if it is an instance of `requests.Response` or `urllib3.HTTPResponse`. These types can be optionally imported (and if the import fails, we know we never need to do this check).
Note that this requires passing `stream=True` when making the request (or `preload_content=False` when using urllib3 directly). This should be noted in the docstring for `Response.stream`.
One way of handling this would be to wrap the object in a closeable stream iterator and then return that to the WSGI server. Here's some proof of concept code (only tested on Python 3):
```py
class CloseableStreamIterator(collections.Iterator):
"""Iterator that wraps a urllib3 response with support for release_conn().
Args:
resp (object): urllib3.HTTPResponse instance.
block_size (int): Number of bytes to read per iteration (default 64K).
decode_content (bool): If True, will attempt to decode the body
according to the Content-Encoding header (default False).
"""
__slots__ = [
'_resp',
'_stream',
]
def __init__(self, resp, block_size=2**16, decode_content=False):
self._resp = resp
self._stream = resp.stream(amt=block_size, decode_content=decode_content)
def __iter__(self):
return self
def __next__(self):
return next(self._stream)
def close(self):
self._resp.release_conn()
```
Contributor guide
Research direction
Start by locating the existing file-like streaming support and the Response.stream docstring. Review how requests.Response and urllib3.HTTPResponse expose streaming and connection cleanup, then verify that streamed responses are accepted and released correctly when streaming is enabled or content preloading is disabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100