element-hq / element-hq/synapse

_MultipartParserProtocol does not properly handle chunked responses

Open
#20,035 0 comments 0 reactions 0 assignees View on GitHub
A-Federation A-Media-Repository T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

### Description

The callbacks that [_MultipartParserProtocol](https://github.com/element-hq/synapse/blob/f61950d54d815b1c11f447bc740a2a2e4f59b522/synapse/http/client.py#L1068) passes to its MultipartParser do not account for the fact that they might be called more than once due to the HTTP response being chunked. Due to this, only the latter part of any value that was split between two chunks makes its way into the MultipartResponse values, leading to errors down the road.

For example, the following media download redirect response:
```
--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448
Content-Type: application/json

{}
--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448
Location: https://cdn.discordapp.com/attachments/1341559314887999501/1533684020762644531/snow-leopard-8096293_1280.png?ex=6a71620d&is=6a70108d&hm=6e2a5760c6376595b76d538f16282f29b603f242d77fa68dfe1a24b59c59766b&

--d38419ab6701bfeb00e1bbe90b037d0656c6f8d6601e9783a1348501b448--
```
Can produce the following error due to the Location URL being split between two chunks:

```
2026-08-03 01:44:07,773 - synapse.http.client - 465 - INFO - GET-9455 - Error sending request to GET 419dcae9&: TypeError argument of type 'NoneType' is not iterable
```

### Steps to reproduce

- Make an authenticated GET request to a homeserver running the latest version of Synapse to the following URL:
- `/_matrix/client/v1/media/download/mautrix.computernewb.com/8J-QiERJU0NPUkQBARKeLRbJBAANFUjAXbpEEIMVSMBdnkIQUNrlSWv0nsD9aV_Ych8SXjY`
- The above media ID is one I know to cause this issue across multiple homeservers, but this can be reproduced with any request that produces a multipart response long enough to be chunked

### Homeserver

computernewb.com

### Synapse Version

v1.157.2

### Installation Method

Debian packages from packages.matrix.org

### Database

PostgreSQL (single server; no; yes)

### Workers

Single process

### Platform

Debian 13 trixie on bare metal, amd64

### Configuration

_No response_

### Relevant log output

```shell
2026-08-03 01:44:07,773 - synapse.http.matrixfederationclient - 1803 - INFO - GET-9455 - {GET-O-33446} [mautrix.computernewb.com] File download redirected, now downloading from: 419dcae9&
2026-08-03 01:44:07,773 - synapse.http.client - 465 - INFO - GET-9455 - Error sending request to GET 419dcae9&: TypeError argument of type 'NoneType' is not iterable
2026-08-03 01:44:07,773 - synapse.media.media_repository - 1066 - ERROR - GET-9455 - Failed to fetch remote media mautrix.computernewb.com/8J-QiERJU0NPUkQBARKeLRbJBAANFUjAXbpEEIMVSMBdnkIQUNrlSWv0nsD9aV_Ych8SXjY
Traceback (most recent call last):
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/media/media_repository.py", line 1021, in _federation_download_remote_file
res = await self.client.federation_download_media(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<8 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/federation/federation_client.py", line 1933, in federation_download_media
return await self.transport_layer.federation_download_media(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/federation/transport/client.py", line 925, in federation_download_media
return await self.client.federation_get_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/matrixfederationclient.py", line 1811, in federation_get_file
length, headers, _, _ = await self._simple_http_client.get_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
str_url, output_stream, max_size
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 735, in get_file
response = await self.request("GET", url, headers=Headers(actual_headers))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 422, in request
request_deferred: defer.Deferred = treq.request(
~~~~~~~~~~~~^
method,
^^^^^^^
...<7 lines>...
**self._extra_treq_args,
^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/treq/api.py", line 169, in request
return _client(kwargs).request(method, url, _stacklevel=3, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/treq/client.py", line 263, in request
d = wrapped_agent.request(
method_, url, headers=headers, bodyProducer=bodyProducer
)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1530, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1623, in request
deferred = self._agent.request(method, uri, headers, bodyProducer)
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/twisted/web/client.py", line 1401, in request
return self._agent.request(
~~~~~~~~~~~~~~~~~~~^
method, uri, actualHeaders, bodyProducer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
).addCallback(self._extractCookies, lastRequest)
^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/synapse/http/client.py", line 318, in request
ip_address = IPAddress(h.hostname) # type: ignore[arg-type]
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/netaddr/ip/__init__.py", line 333, in __init__
self._value = module.str_to_int(addr, flags)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/opt/venvs/matrix-synapse/lib/python3.13/site-packages/netaddr/strategy/ipv4.py", line 118, in str_to_int
if ':' in addr:
^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
2026-08-03 01:44:07,778 - synapse.http.server - 132 - INFO - GET-9455 - SynapseError: 502 - Failed to fetch remote media
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Research direction

Read synapse/http/client.py around _MultipartParserProtocol and inspect how its callbacks receive data from MultipartParser. Reproduce a multipart response with a value split across chunks, then verify that the complete value is retained and the media redirect no longer produces the malformed URL error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.