nextcloud / nextcloud/server

Too many dav UploadCleanup in the job list

Open
#30,772 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

1. to develop feature: dav performance 🚀 technical debt
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

How to use GitHub
  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.
Steps to reproduce
  1. Create a lot of 'apps/dav/lib/BackgroundJob/UploadCleanup' jobs (> 50 000)
  2. Use https://github.com/nextcloud/server/pull/30359 to run these jobs
Expected behaviour

UploadCleanup jobs are only executed less often when not needed

Actual behaviour

After 14 minutes, the cron job stops and there is little difference in the number of jobs since these jobs are only deleted if the files in the upload directory are older than 24 hours or if the upload finishes.

This is causing issues as the same jobs get processed over and over again, wasting resources and potentially blocking other jobs to run.

Server configuration

Database: mysal

PHP version: 8.0

Nextcloud version: master

Primary storage: s3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting apps/dav/lib/BackgroundJob/UploadCleanup and reproduce the behavior with many jobs, using pull request #30359 as described. Trace why jobs remain in the list and compare the observed scheduling with the expected behavior; done means unnecessary UploadCleanup jobs no longer repeatedly consume cron resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.