element-hq / element-hq/synapse
bug: Content-Length is required for media uploads [which is against the spec]
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
### Description
Synapse [REQUIRES the Content-Length to be set](https://github.com/element-hq/synapse/blob/979566ed8fa25511a1d858229c4b8848353c3167/synapse/rest/media/upload_resource.py#L62) when uploading media, which is against the spec, which says that [only Content-Type is required, and only on version 1.11 and under](https://spec.matrix.org/v1.16/client-server-api/#post_matrixmediav3upload)
Why this is actually an issue:
- Content-Length is a protected header in web browsers, which mean it cannot be configured
- Uploading streamed/generated content is quite important for lowering memory usage
The encrypted attachment spec uses AES-CTR, which [I assume] was picked because it can be multi-threaded, thus it can be streamed, this can let a user specify a file, and start uploading the file while streaming the encryption, without pre-computing and without needing to load the entire file into memory, however POST'ing streams in web doesn't set a Content-Length, and since it's a protected header it can't be set manually.
This is also a problem with streaming the uploads of generated content such as video or audio streams which you realistically don't want to retain in-memory, but upload them on the fly.
### Steps to reproduce
- send an authorized POST to /_matrix/media/v3/upload without a Content-Length header
- see instant error
### Homeserver
matrix.org
### Synapse Version
{"server_version":"1.148.0rc1 (b=matrix-org-hotfixes-priv,047f4e920c)"}
### Installation Method
I don't know
### Database
matrix.org so unsure
### Workers
I don't know
### Platform
matrix.org so unsure
### Configuration
_No response_
### Relevant log output
```shell
-/-
```
### Anything else that would be useful to know?
IMHO the best way to solve this is to make the header optional, rather than required
Contributor guide
Research direction
Start at Synapse's media upload endpoint, POST /_matrix/media/v3/upload, and inspect the upload_resource.py code linked in the report, especially the Content-Length check at line 62. Reproduce an authorized upload without that header and compare the behavior with the Matrix v1.16 specification; done means compliant uploads can proceed without Content-Length while preserving upload handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100