nextcloud / nextcloud/backup

Temporary files and other unexpected issues.

Open
#144 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
272
Forks
37
PR merge metrics
No merged PRs in 30d

Description

I did not expect that during first pass all chunks are created in temp-files. The app does not use the temporary directory defined in the config/config.php but the one defined in php.ini under 'sys_temp_dir'. My normal /tmp dir is tmpfs and limited in space. The result was an infinite loop of backup attempts, until I debugged the problem.
Than I tried a full backup with the App-Data directory on a sftp share on another machine in the same rack, since I have > 1 TB of data to backup. I expected that using an external drive would save storage space on the server but then I noticed, that those temporary files created in the temp-dir are not deleted after written to sftp mounted external drive. That lookes like a bug.
In the next attempt, I configured the App-Data folder on the same machine (as external drive on local machine). Now it deletes every temporary chunk after written to App-Data folder as expected.
Btw: I observed that using sftp mounted external drives with other options as "username and password" (e.g. with "Global Credentials") does not work at all with the backup app, although one can mount those drives and write to them. but that looks like another issue.
As you see, I needed more than 3 attempts to get it work (and I think I am not the only one). Thus I tried to get the backup app back to its original state with ./occ backup:reset. Unfortunately that does not reset the app completely to the original state like I expected (like a make clean). The enqueued jobs seems to be still present since the "Restoring Points history" still shows future storing points.
This backup app is way slower than my previous scripted backup procedure but those backups were not with all the meta-data but one big tarball with the complete data inside (beside server-/app-files and database-dump) and a reconstruction was only possible with the complete data exactly in the state is was while creating, not single files. Therefore my first pass only used about 6-8 hours maintenance mode for 1 TB. With this app I need up to15 hours with the same hardware. But at the end of the day, since partial backups use way less resources, this app is my choice for the future.
All in all good work!

(Edit: Typo)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by separating the reported behaviors: temporary files use php.ini's sys_temp_dir instead of config/config.php, cleanup differs for SFTP-mounted App-Data, and ./occ backup:reset leaves queued jobs or history. Reproduce each case and inspect the related backup and reset entry points; done means the intended temporary-directory selection, cleanup, and reset behavior are verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.