Retention does not free disk space

Open
#31,643 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Quiet
Tech stack
mongodb
Domain
databases

Research direction

No source file or test is named. Start by reproducing the retention policy with Rocket.Chat 6.5.3 and MongoDB 5.0.24, then compare the upload collection sizes with the corresponding files under /var/lib/mongodb. Done should identify why deleted uploads do not reduce disk usage and provide a verified fix or a clearly documented limitation.

Written by the indexing model from the issue text.

Description

Description:

Hi, i'm using RocketChat 6.5.3 (mongo wiredTiger), installed manually on Ubuntu LTS.
Retention policy enabled to 180 days and ALL checkboxes are on also – i want to just delete old uploads to save disk space but keep messaging history. If there is a way to fix that 57Gb again to 14Gb then i don't need retention policy at all.

After turning on retention, i see empty messages in the chat history instead of uploads (it's ok) but disk space remains exactly the same even after restart mongo/rocket.

Installation is pretty old (first message was in 2020 with 3.x version) and it took only 14 Gb of space before upgrade. I did upgrade to the latest version (6.5.3) early January this year and now (after a month) it takes whooping 57 Gb with the same users and same activity, which is also too weird for me.

I've also checked collection sizes and got this:

use rocketchat
db.getCollectionNames().forEach((cname) => {
    const csize = db.getCollection(cname).stats(1024 * 1024 * 1024).storageSize;
    if (csize > 1) print(cname + " = " + csize);
});

rocketchat_userDataFiles.chunks = 30.662113189697266
rocketchat_uploads.chunks = 13.659313201904297

But in my disk i have more weird files:

find /var/lib/mongodb/ -type f -size +1G -exec du -h {} +

31G	/var/lib/mongodb/collection-766--3268902447285908541.wt
13G	/var/lib/mongodb/collection-14-7180290947299615862.wt
14G	/var/lib/mongodb/collection-0-5684180545419248738.wt
Steps to reproduce:
  1. Install Rocketchat
  2. Turn on retention policy
  3. Check disk usage
Expected behavior:

There is no huge weird files in mongo lib and retention policy cleans up space when deleting uploaded files.

Actual behavior:

There are big files that are probably have no use and retention policy don't clean up the disk space.

Server Setup Information:
  • Version of Rocket.Chat Server: 6.5.3
  • Operating System: Ubuntu 20.04 LTS
  • Deployment Method: tar
  • Number of Running Instances: 1
  • DB Replicaset Oplog: idk ???
  • NodeJS Version: 14.21.3
  • MongoDB Version: 5.0.24
Client Setup Information
  • Desktop App or Browser Version: any
  • Operating System: any
Additional context

Described above.

Relevant logs:

Ask please what is needed.

Dominant language
TypeScript
Stars
46.1k
Forks
13.9k
Avg merge
3d 3h
Merged PRs (30d)
130

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from RocketChat/Rocket.Chat

All issues in RocketChat/Rocket.Chat

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.