nextcloud / nextcloud/server

[Bug]: Extremely slow trashbin restore

Open
#52,268 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 30-feedback bug feature: trashbin performance 🚀
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

Upon synchronization my Windows share I made an error in the exclude list..
That caused the Linux server to become slow. This triggered me in realizing the faulty sync. Then some 40.000 files were already deleted!

I went to the web-interface to start restoring.. and discovered that that also was an extremely server intensive activity.

Steps to reproduce
  1. Delete many thousands of files on your Windows share
  2. Restore them via the web-interface
  3. See the "show full processlist;"
Expected behavior

Since I use AWS-S3 I expected the restore to be a simple MySql update of the folder entry..

Nextcloud Server version

30

Operating system

RHEL/CentOS

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report
-
List of activated Apps
-
Nextcloud Signing status
-
Nextcloud Logs
-
Additional info

With the command "show full processlist;"

I saw that these were the queries I saw several of that were running a multitude of seconds.

SELECT filecache.fileid, storage, path, path_hash, filecache.parent, filecache.name, mimetype, mimepart, size, mtime
, storage_mtime, encrypted, etag, filecache.permissions, checksum, unencrypted_size, metadata_etag, creation_time, upload_time
, meta.json AS meta_json, meta.sync_token AS meta_sync_token FROM oc_filecache filecache
LEFT JOIN oc_filecache_extended fe ON filecache.fileid = fe.fileid
LEFT JOIN oc_files_metadata meta ON filecache.fileid = meta.file_id
WHERE (filecache.parent = 88189) AND (storage = 2) ORDER BY name ASC

id 88189 is the ID of the trashbin. So it appears that after each delete the new remaining lis of files in the Trashbin is read.

But with my still some 40.000 files remaining in the trashbin this is a big query! I expect things would dramaticaly spead up when the MOVE / RESTORE function become separate from the "show trashbin content".

In other words, why not just first simply RESTORE everything I told it to.. and then.. all at the end read what's left in the trash?

If that could be done, the operations in the trashbin will likely bee only slighly slower when restoring just one file (since the restore call will be split into two operations). But when the RESTORE (or the permanently delete?) action is done on a multitude of files I expect the response will be far faster and the load on the server far lower!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the web-interface trashbin restore flow and the filecache query shown in the issue, using the reported trashbin parent ID and MariaDB process list as the reproduction signals. Compare restoring many files with the trashbin contents being refreshed after each operation. Done means bulk restore no longer repeatedly reads the full remaining trashbin list, with the reported server load and query behavior improved.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, mariadb, php
Domain
backend, databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.