nextcloud / nextcloud/server

[Bug]: files_reminders causing N+1 queries for folders with more than 512 elements

Open
#61,193 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

1. to develop 35-feedback bug feature: dav feature: files_reminders performance 🚀
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description
Summary

OCA\FilesReminders\Dav\PropFindPlugin preloads reminders information for an entire collection during PROPFIND requests. ReminderService has a cacheFolder method which uses CappedMemoryCache as cache. The problem is that the default size for this cache is 512 meaning that if a directory has more than 512 files inside we start getting cache misses and start querying the DB.

What's worse is that once the condition hits, it always voids the pre-cached data and causes N+1 queries because:

  • the preload mechanism starts caching reminders for all files in a directory
  • once the limit of the CappedMemoryCache is reached, the information that was cached earlier is discarded
  • when the PROPFIND is serialized into XML, the first file is no longer in the cache, causing a query for the single file
  • the result is cached, pushing another older result out of the cache
Steps to reproduce
  1. Create a folder
  2. Upload more than 512 files
  3. Do a PROPFIND request on the folder
  4. See that the code runs >512 queries targeting the reminders table
Expected behavior

Queries are executed in N+1 way only if there are more than 512 reminders in files in a folder.

Nextcloud Server version

35 (master)

Operating system

None

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OCA\FilesReminders\Dav\PropFindPlugin and ReminderService, then inspect how CappedMemoryCache is used during PROPFIND serialization. Reproduce the case with more than 512 files and measure queries against the reminders table; done means the preload no longer causes per-file queries after the cache limit is reached.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.