nextcloud / nextcloud/groupfolders

Group folders dramatically slows down my server.

Open
#2,256 1 comment 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage bug performance :rocket: technical debt
Dominant language
PHP
Stars
343
Forks
106
Avg merge
2d 3h
Merged PRs (30d)
34

Description

I have a large nextcloud setup (around 5TB) with most folders primarily group folders. Basically all the files and folders are shared across the company but depending on your group you get different permission per file. I have custom software that connects to nextcloud and using the api it sets the different permission per folders. After using that the setup for a few weeks nextcloud slowed down dramatically and we upgraded our server to a super server just to keep things moving. I did an sql analysis on the number of queries it makes just to get the root path and with group folders plugin enabled it makes around 12000 queries! vs. disabled it makes around 500. I also found one query that was extremely slow and took about 8 seconds to run, please see below. Are we doing something wrong in the setup or is group folders not made to handle so many files?

SELECT `file`.`fileid`, `storage`, `path`, `path_hash`, `file`.`parent`, `file`.`name`, `mimetype`, `mimepart`, `size`, 
`mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `unencrypted_size` 
FROM `oc_filecache` `file` 
WHERE (((`mimetype` <> 2) OR (`size` = 0)) AND (`mtime` > 1674421392)) AND (((`storage` = 2) 
AND ((`path` = 'files') OR (`path` LIKE 'files/%'))) OR ((`storage` = 68) 
AND ((`path_hash` = '570bcc05900b9eb600ba5e918520df78') OR (`path` LIKE '__groupfolders/461/%'))) OR ((`storage` = 68) 
AND ((`path_hash` = 'fbd7c214a3bd37098d045cf59bddc551') OR (`path` LIKE '__groupfolders/361/%'))) OR ((`storage` = 68) 
 AND ((`path_hash` = '959c98e248c3c3afefaeceaf803169d6') OR (`path` LIKE '__groupfolders/584/%'))) OR ((`storage` = 68)
 AND ((`path_hash` = '2b437574d7f2256ce0c8bffb1d3f7577') OR (`path` LIKE '__groupfolders/585/%'))) OR ((`storage` = 68) 
 AND ((`path_hash` = '0866597a092404891d9b01184f271c83') OR (`path` LIKE '__groupfolders/586/%'))) OR ((`storage` = 68) ... this keeps going for a while.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported root-path API request with group folders enabled and compare its SQL query count and duration with group folders disabled. Trace the group folders path that builds the long oc_filecache query, then identify a measurable way to reduce the reported query load without changing permissions. Done means the large setup no longer shows the reported slowdown under the same comparison.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.