nextcloud / nextcloud/server

[Bug]: Inefficient comments table access

Open
#51,983 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 29-feedback bug feature: database performance 🚀
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

I found queries like this in production:

SELECT
  *
FROM
  `oc_comments`
WHERE
  `object_type` = 'files'
ORDER BY
  `creation_timestamp` DESC,
  `id` DESC
LIMIT
  100 OFFSET 200

The query can use the index comments_object_index but only to filter for the object_type. The rest has to be fetched from the table. This is relatively expensive, because the query needs to scan 27x number of actually returned rows:

Image

Moreover, the limit/offset doesn't scale. To retrieve 100 results with an offset of 200, 300 rows have to be built.

This is also a bug because as the data changes, queries might miss entries for new inserts or see overlaps for deleted rows.

Steps to reproduce
  1. Install Nextcloud
  2. ???
Expected behavior

This can be achieved by two changes

  1. Change the query to use a timestamp for relative comparison of where the results should start
  2. Add an index on (object_type, creation_timestamp, id)

This will allow the database to go to the start of the result directly without scanning rows that are not returned. Adding the id to the index helps Postgres and any other databases that use a heap table.

Nextcloud Server version

29

Operating system

None

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

https://use-the-index-luke.com/sql/partial-results/fetch-next-page

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names the oc_comments query, comments_object_index, and the requested (object_type, creation_timestamp, id) index, but no source files or tests. Locate the comments query and database migration entry points first, then inspect how pagination is implemented. Done means keyset pagination and the covering index are implemented with tests or query-plan verification.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.