element-hq / element-hq/synapse
_MultipartParserProtocol does not properly handle chunked responses
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
### Description
The callbacks that [_MultipartParserProtocol](https://github.com/element-hq/synapse/blob/f61950d54d815b1c11f447bc740a2a2e4f59b522/synapse/http/client.py#L1068) passes to its MultipartParser do not account for the fact that they might be called more than once due to the HTTP response being chunked. Due to this, only the latter part of any value that was split between two chunks makes its way into the MultipartResponse values, leading to errors down the road.
For example, the following media download redirect response:
```
--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448
Content-Type: application/json
{}
--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448
Location: https://cdn.discordapp.com/attachments/1341559314887999501/1533684020762644531/snow-leopard-8096293_1280.png?ex=6a71620d&is=6a70108d&hm=6e2a5760c6376595b76d538f16282f29b603f242d77fa68dfe1a24b59c59766b&
--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448--
```
Can produce the following error due to the Location URL being split between two chunks:
```
2026-08-03 01:44:07,773 - synapse.http.client - 465 - INFO - GET-9455 - Error sending request to GET 419dcae9&: TypeError argument of type 'NoneType' is not iterable
```
### Steps to reproduce
- Make an authenticated GET request to a homeserver running the latest version of Synapse to the following URL:
- `/_matrix/client/v1/media/download/mautrix.computernewb.com/8J-QiERJU0NPUkQBARKeLRbJBAANFUjAXbpEEIMVSMBdnkIQUNrlSWv0nsD9aV_Ych8SXjY`
- The above media ID is one I know to cause this issue across multiple homeservers, but this can be reproduced with any request that produces a multipart response long enough to be chunked
### Homeserver
computernewb.com
### Synapse Version
v1.157.2
### Installation Method
Debian packages from packages.matrix.org
### Database
PostgreSQL (single server; no; yes)
### Workers
Single process
### Platform
Debian 13 trixie on bare metal, amd64
### Configuration
_No response_
### Relevant log output
```shell
2026-08-03 01:44:07,773 - synapse.http.matrixfederationclient - 1803 - INFO - GET-9455 - {GET-O-33446} [mautrix.computernewb.com] File download redirected, now downloading from: 419dcae9&
2026-08-03 01:44:07,773 - synapse.http.client - 465 - INFO - GET-9455 - Error sending request to GET 419dcae9&: TypeError argument of type 'NoneType' is not iterable
2026-08-03 01:44:07,773 - synapse.media.media_repository - 1066 - ERROR - GET-9455 - Failed to fetch remote media mautrix.computernewb.com/8J-QiERJU0NPUkQBARKeLRbJBAANFUjAXbpEEIMVSMBdnkIQUNrlSWv0nsD9aV_Ych8SXjY
Traceback (most recent call last):
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/media/media_repository.py", line 1021, in _federation_download_remote_file
res = await self.client.federation_download_media(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<8 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/federation/federation_client.py", line 1933, in federation_download_media
return await self.transport_layer.federation_download_media(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/federation/transport/client.py", line 925, in federation_download_media
return await self.client.federation_get_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/matrixfederationclient.py", line 1811, in federation_get_file
length, headers, _, _ = await self._simple_http_client.get_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
str_url, output_stream, max_size
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 735, in get_file
response = await self.request("GET", url, headers=Headers(actual_headers))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 422, in request
request_deferred: defer.Deferred = treq.request(
~~~~~~~~~~~~^
method,
^^^^^^^
...<7 lines>...
**self._extra_treq_args,
^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/treq/api.py", line 169, in request
return _client(kwargs).request(method, url, _stacklevel=3, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/treq/client.py", line 263, in request
d = wrapped_agent.request(
method_, url, headers=headers, bodyProducer=bodyProducer
)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1530, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1623, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1401, in request
return self._agent.request(
~~~~~~~~~~~~~~~~~~~^
method, uri, actualHeaders, bodyProducer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
).addCallback(self._extractCookies, lastRequest)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 318, in request
ip_address = IPAddress(h.hostname) # type: ignore[arg-type]
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/netaddr/ip/__init__.py", line 333, in __init__
self._value = module.str_to_int(addr, flags)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/netaddr/strategy/ipv4.py", line 118, in str_to_int
if ':' in addr:
^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
2026-08-03 01:44:07,778 - synapse.http.server - 132 - INFO - GET-9455 - SynapseError: 502 - Failed to fetch remote media
```
### Anything else that would be useful to know?
_No response_
Contributor guide
Research direction
Read synapse/http/client.py around _MultipartParserProtocol and inspect how its callbacks receive data from MultipartParser. Reproduce a multipart response with a value split across chunks, then verify that the complete value is retained and the media redirect no longer produces the malformed URL error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100