Long processing times while uploading to S3
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
Running Nextcloud 17.0.3 behind Cloudflare works fine, unless the upload processing time takes longer than 100s (in our case, uploading files bigger than 2GB trough WebUI to a S3 primary backend). In that case, Cloudflare returns a 524.
Error when assembling chunks, status code 524
Contacting Cloudflare about this hits on a:
... modify your process so it sends a keep alive request...
Cloudflare will typically wait for a HTTP response from origin server for 100 seconds. If no response (or <8Kb) is sent by the origin in that time, Cloudflare will close the connection and serve a 524 error page.
So the question is, can (should?) Nextcloud detect those long running processing times and send a keepalive request every (say) 60 seconds to avoid these problems on some proxies like Cloudflare?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the WebUI upload chunk-assembly path when S3 is the primary backend, and reproduce the failure with files larger than 2GB through Cloudflare. Investigate whether the long-running request can emit a keepalive before Cloudflare's 100-second limit; done means the upload completes without the reported 524 or the limitation is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, php
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100