[Bug]: files_reminders causing N+1 queries for folders with more than 512 elements
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
Summary
OCA\FilesReminders\Dav\PropFindPlugin preloads reminders information for an entire collection during PROPFIND requests. ReminderService has a cacheFolder method which uses CappedMemoryCache as cache. The problem is that the default size for this cache is 512 meaning that if a directory has more than 512 files inside we start getting cache misses and start querying the DB.
What's worse is that once the condition hits, it always voids the pre-cached data and causes N+1 queries because:
- the preload mechanism starts caching reminders for all files in a directory
- once the limit of the
CappedMemoryCacheis reached, the information that was cached earlier is discarded - when the PROPFIND is serialized into XML, the first file is no longer in the cache, causing a query for the single file
- the result is cached, pushing another older result out of the cache
Steps to reproduce
- Create a folder
- Upload more than 512 files
- Do a PROPFIND request on the folder
- See that the code runs >512 queries targeting the reminders table
Expected behavior
Queries are executed in N+1 way only if there are more than 512 reminders in files in a folder.
Nextcloud Server version
35 (master)
Operating system
None
PHP engine version
None
Web server
None
Database engine version
None
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
Additional info
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OCA\FilesReminders\Dav\PropFindPlugin and ReminderService, then inspect how CappedMemoryCache is used during PROPFIND serialization. Reproduce the case with more than 512 files and measure queries against the reminders table; done means the preload no longer causes per-file queries after the cache limit is reached.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100