[BOUNTY $1,000] Can Not Upload Files Larger Than CDN or Proxy Single Request Limits without Chunking

Description

Since the change to send files via the REST API https://github.com/RocketChat/Rocket.Chat/pull/16617, RocketChat instances hosted behind a CDN can not upload files larger than the limit for a single request (100MB for Cloudflare)

Problem has been recognized in posts here: https://github.com/RocketChat/Rocket.Chat/pull/16617#issuecomment-591119408, https://forums.rocket.chat/t/file-uploads-are-super-slow-because-theyre-sent-in-many-small-post-requests-wtf/4600/24, and https://forums.rocket.chat/t/file-uploads-are-super-slow-because-theyre-sent-in-many-small-post-requests-wtf/4600/26

While the previous uploading library https://github.com/jalik/meteor-jalik-ufs#uploading-files had issues, it did chunk uploads. If the upload chunk size was configurable, rather than a single POST request it would fix this issue and allow administrators to tune the setup to their network.

Steps to reproduce

Host RocketChat instance behind Cloudflare, and upload a file over 100MB

Server Setup Information

Version of Rocket.Chat Server: 3.14.3
Operating System: Ubuntu
Deployment Method: Manual install
Number of Running Instances: 4
DB Replicaset Oplog: Enabled
NodeJS Version: v12.18.4
MongoDB Version: 4.0.25

Client Setup Information

Desktop App or Browser Version: Firefox version 100
Operating System: Any

Additional context

Relevant logs

https://app.bountysource.com/issues/109087930-bounty-can-not-upload-files-larger-than-cdn-or-proxy-single-request-limits-without-chunking [$1,000]

Start by tracing Rocket.Chat's REST API file-upload entry point and the current single-request path described in the issue. Reproduce the failure with a file over the CDN or proxy limit, then verify that uploads use configurable chunks and that files larger than 100MB succeed behind Cloudflare.