RocketChat / RocketChat/Rocket.Chat
Large "Download My Data" export crashes RC
@gabrieleiro is already working on this.
Since Apr 27, 2021.
- #21898 by @gabrieleiro — closed without merging
- Dominant language
- TypeScript
- Stars
- 46.1k
- Forks
- 13.9k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 130
Description
Description:
When "Download My Data" is initiated for a user who has an excessive amount of historical data, the RC service will crash while trying to compile an export archive.
Steps to reproduce:
- Click on "Download My Data" for a user with a large amount of historical data in the system.
- Observe that eventually the system will grind to a halt.
Expected behavior:
The archive should be produced and a link should be emailed to the user.
Actual behavior:
The export job tries to gather all historical data into an archive, but the volume of data eventually brings down the system. My assumption is that this is due to the usage of /tmp or tmpfs as the workspace for compiling "Download My Data" archives. I have not tested it yet, but I imagine a channel export of a channel with excessive amounts of data (assets, etc.) would also produce a problem.
Server Setup Information:
- Version of Rocket.Chat Server: 3.6.0
- Operating System: RedHat
- Deployment Method: tar file
- Number of Running Instances: 3
- DB Replicaset Oplog:
- NodeJS Version:
- MongoDB Version:
Client Setup Information
- Desktop App or Browser Version: Chrome
- Operating System:
Additional context
Our sever has 2.0G of space allocated for tmpfs. Ideally, the temporary area used by the export functions to compile archives of exported data would take place in a much larger area. I'm not an expert in tmpfs, so please correct me if I am wrong, but I believe the max of what you can allocate to it is limited by your server's RAM, and that's really not enough to support one user with a lot of data, much less any concurrent exports that may take place.
Relevant logs:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.