nextcloud / nextcloud/server

twice running cron.php / GenerateMetadataJob

Open
#43,184 3 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 28-feedback enhancement needs review
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

Due to problems with the endlessly running cron.php, I identified the issue by finding the "long running job." By obtaining the start time of the PID from ps and searching the logs, I discovered that the issue lies with the "GenerateMetadataJob," which is hanging. This job starts but never receives a "Finished" event:

{"reqId":"T7WSKkyswz0F035LClQ6","level":0,"time":"2024-01-27T10:00:03+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"Run OC\Core\BackgroundJobs\GenerateMetadataJob job with ID 36764","userAgent":"--","version":"28.0.1.1","data":{"app":"cron"}}

Additionally, the job is running twice or more:

[picard@borgcube ~]$ ps aux | grep -E "^http.*?cron.php$"
http     1944203 97.1  0.0 428316  4072 ?        R    Jan27 2821:33 php -f /srv/http/nextcloud/cron.php
http     1999635 96.8  0.2 428316 23768 ?        R    Jan28 1414:10 php -f /srv/http/nextcloud/cron.php
http     2055241 98.2  0.9 428316 75584 ?        R    11:05  19:55 php -f /srv/http/nextcloud/cron.php

[picard@borgcube ~]$ ps -p 1944203 -o lstart
                 STARTED
 Sat Jan 27 11:00:01 2024
[picard@borgcube ~]$ ps -p 1999635 -o lstart
                 STARTED
 Sun Jan 28 11:05:01 2024
[picard@borgcube ~]$ ps -p 2055241 -o lstart
                 STARTED
 Mon Jan 29 11:05:01 2024

I'm not sure why this is happening, but I have an idea that it might be related to symlinks and/or multiple mounted devices.

Additionally, while investigating this issue, I read some code and would like to recommend changing the "funny if/else construct" for checking "onlyTimeSensitive" at line 122ff in cron.php with this single line:

$onlyTimeSensitive = ($currentHour >= $startHour) == ($currentHour > $endHour);

Thanks a lot!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cron.php and the OC\Core\BackgroundJobs\GenerateMetadataJob entry shown in the issue. Compare the process start times from ps with the cron logs, focusing on why the job starts without a Finished event and why multiple cron.php processes remain active. The suggested onlyTimeSensitive expression is also located around line 122, but the issue does not define an accepted fix or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.