nextcloud / nextcloud/backup

Backup stuck in first pass

Open
#106 7 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
272
Forks
37
PR merge metrics
No merged PRs in 30d

Description

I've tried to make a backup by setting both app_data and restoring points to the same external storage (sftp), both in sub-folders. I don't have enough free disk space to duplicate all the data locally.

I ran a manual full-backup through the web UI first and waited about 15 minutes for some data to appear on the remote server. It never got past 2.8MB so I assumed the process failed. These are the files and folders that got created remotely.

20211207233004-full-rVdbX4adce6TfYy/
├── app.zip
├── data
│   └── data-17a75f77-d188-450a-aac5-0e3fd8790954
└── restore.php

I tried again this time by running occ backup:point:create and letting it run mostly unattended for roughly 2 hours. The same folder structure got created with again no data. I noticed the docker container ran out of space and the occ command was consuming 100% of a single cpu core. Occ was not consuming much cpu at the start of the process, I don't know when this started.

I was unable to log into the docker container so I could not inspect the offending files. I know it must have been outside /var/www/html because that's mounted to another filesystem. After recreating the container (flushing docker overlayfs), about 10,6GB of space was returned.

This is the output from occ:

www-data@c734ca92f6b1:~/html$ php occ backup:point:create
> maintenance mode: on
> initialization of the AppData
> initialization of the RestoringPoint: 20211207234018-full-ocppBxgNN8fWWvH based on NC23.0.0.10
> initialization of the storage
> preparation of the data to be stored in the restoring point
> preparation of internal data
> creating chunks
  * data: /var/www/html/data/, 58019 files

Maybe backup choked on the number of files?

I'm running nc v23.0.0 with backup app v1.0.0 through the official docker image with MySQL(mariadb v10.7) as database.

I may try again soon to see if I can get some more info on that disk usage thing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the occ backup:point:create entry point and the output showing creating chunks for /var/www/html/data/; compare that path with the Docker-mounted storage and the SFTP sub-folders. Reproduce with the reported Nextcloud 23.0.0, backup app 1.0.0, and roughly 58,019 files, then identify why the first pass stalls or consumes container space. Done means the cause is isolated and a focused fix or actionable diagnostic is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, mariadb, php
Domain
backend, databases, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.