nextcloud / nextcloud/backup

S3 upload still failes with OOM for 1.0.6

Open
#237 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
272
Forks
37
PR merge metrics
No merged PRs in 30d

Description

The latest release still has the OOM bug when uploading to remote storage.

  • nextcloud:23.0.2 with postgrest client added
  • postgres:10.3

image

  > Health status: 0 correct, 363 missing and 0 faulty files
  * Uploading data/data-e1a62938-57ed-4033-ab61-48b258866447/00001-4J8cz2vkcCkabEn: ok
  * Uploading data/data-e1a62938-57ed-4033-ab61-48b258866447/00002-KNuSXKImJx65wKm: PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 104857824 bytes) in /var/www/html/lib/private/Files/Storage/Common.php on line 208
PHP Fatal error:  Uncaught Error: Class "OC\Log\ExceptionSerializer" not found in /var/www/html/lib/private/Log.php:311
Stack trace:
#0 /var/www/html/lib/private/Log/ErrorHandler.php(93): OC\Log->logException(Object(Error), Array)
#1 [internal function]: OC\Log\ErrorHandler::onError(2, 'fopen(/tmp/oc_t...', '/var/www/html/a...', 641)
#2 /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php(641): fopen('/tmp/oc_tmp_6Qo...', 'r')
#3 /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php(501): OCA\Files_External\Lib\Storage\AmazonS3->writeBack('/tmp/oc_tmp_6Qo...', 'backups/2022032...')
#4 [internal function]: OCA\Files_External\Lib\Storage\AmazonS3->OCA\Files_External\Lib\Storage\{closure}()
#5 /var/www/html/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php(119): call_user_func(Object(Closure))
#6 [internal function]: Icewind\Streams\CallbackWrapper->stream_close()
#7 {main}
  thrown in /var/www/html/lib/private/Log.php on line 311

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with apps/files_external/lib/Lib/Storage/AmazonS3.php, especially the writeBack path shown in the trace, and compare it with lib/private/Files/Storage/Common.php where the memory exhaustion occurs. Reproduce the upload using the listed Nextcloud and PostgreSQL versions, then verify that the affected files upload without the PHP out-of-memory failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.