getsentry / getsentry/responses

responses multipart_matcher does not consume the passed iterator; how to mock?

Open
#713 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
4.3k
Forks
378
Avg merge
4d 1h
Merged PRs (30d)
1

Description

### Describe the bug

Hi, I am developing some python code utilizing mulipart/form-data headers for uploading a file. When I try to mock the endpoint response with the following code, then it hangs.
```
@responses.activate
def test_multipart_upload(client, test_file_definition):
responses.add(
responses.POST,
url=gen_url(),
json=1,
)
client.multipart_upload(...)
```

Responses: 0.25.0
Python: 3.11

### Additional context

Once I will add a multipart matcher, I can get an error stating that the iterator does not match the data (as the generator is just an object in the memory).
```
~#@❯ pytest -x --no-cov -k test_multipart_upload
Test session starts (platform: win32, Python 3.11.8, pytest 7.4.0, pytest-sugar 1.0.0)
rootdir: C:\Users\
configfile: pytest.ini
testpaths: tests
plugins: anyio-4.3.0, nbmake-1.5.3, cov-4.1.0, icdiff-0.9, sugar-1.0.0, timeout-2.3.1

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_create_new_version ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

client = Client(dummy@unknown(http://test:54422))>
tmp_path = WindowsPath('C:/Users//pytest-442/test_multipart_upload0')

@responses.activate
def test_multipart_upload(client, tmp_path):
file_path = tmp_path / "test_data.txt"
write_file(file_path) # 1 kB

responses.add(
responses.POST,
url=gen_url(),
match=[
responses.matchers.multipart_matcher({"file_name": b""})
],
json=1,
)
> client.multipart_upload(...)

C:\Users\\tests\client\test_multipart_upload.py
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Users\\envs\mylib311\Lib\site-packages\responses\__init__.py:1173: in send
return self._on_request(adapter, request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = , adapter = , request = , retries = None
kwargs = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'timeout': None, ...}, match = None
match_failed_reasons = ['multipart/form-data doesn\'t match. Request body differs. ' status=200 content_type='application/json' headers='null')>

def _on_request(
self,
adapter: "HTTPAdapter",
request: "PreparedRequest",
*,
retries: Optional["_Retry"] = None,
**kwargs: Any,
) -> "models.Response":
# add attributes params and req_kwargs to 'request' object for further match comparison
# original request object does not have these attributes
request.params = self._parse_request_params(request.path_url) # type: ignore[attr-defined]
request.req_kwargs = kwargs # type: ignore[attr-defined]
request_url = str(request.url)

match, match_failed_reasons = self._find_match(request)
resp_callback = self.response_callback

if match is None:
if any(
[
p.match(request_url)
if isinstance(p, Pattern)
else request_url.startswith(p)
for p in self.passthru_prefixes
]
):
logger.info("request.allowed-passthru", extra={"url": request_url})
return self._real_send(adapter, request, **kwargs) # type: ignore

error_msg = (
"Connection refused by Responses - the call doesn't "
"match any registered mock.\n\n"
"Request: \n"
f"- {request.method} {request_url}\n\n"
"Available matches:\n"
)
for i, m in enumerate(self.registered()):
error_msg += "- {} {} {}\n".format(
m.method, m.url, match_failed_reasons[i]
)

if self.passthru_prefixes:
error_msg += "Passthru prefixes:\n"
for p in self.passthru_prefixes:
error_msg += f"- {p}\n"

response = ConnectionError(error_msg)
response.request = request

self._calls.add(request, response)
> raise response
E requests.exceptions.ConnectionError: Connection refused by Responses - the call doesn't match any registered mock.
E
E Request:
E - POST http://test:54422/?complete=False&encoding=utf-16
E
E Available matches:
E - POST http://test:54422/ multipart/form-data doesn't match. Request body differs. aren't equal b'--a5d476d4-c8cc-43cc-bf88-6b8246dadfa1\r\nContent-Disposition: form-data; name="file_name"; filename="file_name"\r\n\r\n\r\n--a5d476d4-c8cc-43cc-bf88-6b8246dadfa1--\r\n'

C:\Users\\envs\mylib311\Lib\site-packages\responses\__init__.py:1100: ConnectionError
```

### Version of `responses`

0.25.0

### Steps to Reproduce

Add an generator to be consumed by the request.

### Expected Result

Code using responses does not hang anymore.

### Actual Result

Tests using responses hang.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.