nextcloud / nextcloud/backup

Upload fails due to out of memory

Open
#244 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
272
Forks
37
PR merge metrics
No merged PRs in 30d

Description

Hi all

Uploading a point never really worked in my setup. The process stops after a couple minutes. The upload during the background processing of nextcloud does not work too.
As I found out today, its because of the PHP going out of memory.
I started the upload process manually for example with the following parameters for php:

(I started it inside the docker container)

Limit to 4GB of memory

php --define apc.enable_cli=1 --define memory_limit=4096M /var/www/html/occ backup:point:upload 20220330174131-full-PRHhBIchcGk5AJl
Result:

...
  * Uploading data/data-9b2543e5-06cf-4140-a1cb-790b68c829be/00017-j2dPiFe6UoPaGAR: ok
  * Uploading data/data-9b2543e5-06cf-4140-a1cb-790b68c829be/00018-QAsOQFCZ4u3E5sd: PHP Fatal error:  Allowed memory size of 4294967296 bytes exhausted (tried to allocate 139810208 bytes) in /var/www/html/custom_apps/backup/lib/Service/PackService.php on line 606
No memory limit at all

php --define apc.enable_cli=1 --define memory_limit=-1 /var/www/html/occ backup:point:upload 20220330174131-full-PRHhBIchcGk5AJl
Result (because of the "unlimited" memory it used more than 8GB of memory):

...
  * Uploading data/data-3daeea38-1fc4-4a73-a076-4d48e4090fac/00018-L3NjJfWfK2fiLQV: ok
  * Uploading data/data-3daeea38-1fc4-4a73-a076-4d48e4090fac/00019-pNi5zxKK6LpRXME: ok
  * Uploading data/data-3daeea38-1fc4-4a73-a076-4d48e4090fac/00020-OvB968OSmGeU3BD: Killed

Is there a memory leak somewhere? How could I prevent this?

Thx for the support

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with custom_apps/backup/lib/Service/PackService.php at line 606 and reproduce the failure using the reported occ backup:point:upload command inside the Docker container. Compare memory use with the 4096M and unlimited PHP settings while processing the affected point; done means the upload no longer exhausts available memory or is killed.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, php
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.