nextcloud / nextcloud/cookbook

Reindex in steps

Open
#613 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Backend database enhancement php
Dominant language
HTML
Stars
641
Forks
113
Avg merge
21h 31m
Merged PRs (30d)
26

Description

Is your feature request related to a problem? Please describe.
Currently, the reindex of the JSON files happens on a regular time schedule (defined by a config variable).

As the instance is growing, this can lead to problems once the reindex takes longer than the timeout of the HTTP server, and no page of the cookbook app can be served anymore in time. Additionally, the app gets laggy in these cases.

Describe the solution you'd like
The ideal solution consists of two parts:

  1. A background job is used to check every 5 min if any recipes have to be reindexed (of any user).
  2. The reindex is limited to a certain number of recipes to index. Something like 50 or 100 recipes maximum. The recipes should age (date of last reindex in DB) and once the recipe is old enough a reindex could take place.
    Due to the regular checks (see point 1) the recipes will soon be indexed completely.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the current scheduled reindex and the configuration variable that controls its timing. Define completion as a background check that reindexes bounded batches, tracks recipe reindex age in the database, and allows all recipes to become indexed over successive checks.

Written by the indexing model from the issue text.

Assessment

Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.