Files are deleted during sync even if errors were encountered

The associated forum post URL from https://forum.rclone.org

https://forum.rclone.org/t/how-to-deal-with-http-error-524-that-causes-deletion-of-good-files/

What is the problem you are having with rclone?

During a dry run of a sync from http remote protected by cloudflare, rclone reports that it is going to remove files which are actually present on the remote. This happens because cloudflare returns http error 524 to an http client when web server fails to return response to a request within some fixed period of time to cloudflare. When rclone gets such a response it decides that a file that it tried to stat during this request doesn't exist on the remote, and therefore deletes it.

Instead of deleting files rclone should either retry a request (and therefore eventually succeed to stat the file) or treat such a return code as an error (and therefore not delete files at the end of an operation, as it is written in 'rclone sync' documentation, which says: "Note that files in the destination won't be deleted if there were any errors at any point")

What is your rclone version (output from rclone version)

rclone v1.60.0

• os/version: arch (64 bit)
• os/kernel: 6.0.8-arch1-1 (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.19.2
• go/linking: dynamic
• go/tags: none

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Linux, 64 bit

Which cloud storage system are you using? (e.g. Google Drive)

http web remote protected by cloudflare

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

rclone --config="" -vv sync --dry-run --http-url

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

https://pastebin.com/bXw99h4e

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Start with the sync command's handling of errors from the HTTP remote and reproduce the dry-run using an HTTP 524 response. Verify that a failed stat is retried or recorded as an error, and that destination files are not deleted after the operation reports an error.