nextcloud / nextcloud/groupfolders
Group folders dramatically slows down my server.
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 343
- Forks
- 106
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 34
Description
I have a large nextcloud setup (around 5TB) with most folders primarily group folders. Basically all the files and folders are shared across the company but depending on your group you get different permission per file. I have custom software that connects to nextcloud and using the api it sets the different permission per folders. After using that the setup for a few weeks nextcloud slowed down dramatically and we upgraded our server to a super server just to keep things moving. I did an sql analysis on the number of queries it makes just to get the root path and with group folders plugin enabled it makes around 12000 queries! vs. disabled it makes around 500. I also found one query that was extremely slow and took about 8 seconds to run, please see below. Are we doing something wrong in the setup or is group folders not made to handle so many files?
SELECT `file`.`fileid`, `storage`, `path`, `path_hash`, `file`.`parent`, `file`.`name`, `mimetype`, `mimepart`, `size`,
`mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `unencrypted_size`
FROM `oc_filecache` `file`
WHERE (((`mimetype` <> 2) OR (`size` = 0)) AND (`mtime` > 1674421392)) AND (((`storage` = 2)
AND ((`path` = 'files') OR (`path` LIKE 'files/%'))) OR ((`storage` = 68)
AND ((`path_hash` = '570bcc05900b9eb600ba5e918520df78') OR (`path` LIKE '__groupfolders/461/%'))) OR ((`storage` = 68)
AND ((`path_hash` = 'fbd7c214a3bd37098d045cf59bddc551') OR (`path` LIKE '__groupfolders/361/%'))) OR ((`storage` = 68)
AND ((`path_hash` = '959c98e248c3c3afefaeceaf803169d6') OR (`path` LIKE '__groupfolders/584/%'))) OR ((`storage` = 68)
AND ((`path_hash` = '2b437574d7f2256ce0c8bffb1d3f7577') OR (`path` LIKE '__groupfolders/585/%'))) OR ((`storage` = 68)
AND ((`path_hash` = '0866597a092404891d9b01184f271c83') OR (`path` LIKE '__groupfolders/586/%'))) OR ((`storage` = 68) ... this keeps going for a while.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported root-path API request with group folders enabled and compare its SQL query count and duration with group folders disabled. Trace the group folders path that builds the long oc_filecache query, then identify a measurable way to reduce the reported query load without changing permissions. Done means the large setup no longer shows the reported slowdown under the same comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100