Add a flag to skip source files deleted during a sync

Before you start

• I have searched the forum and the existing issues and this hasn't already been requested.
• I have checked the latest beta and this feature doesn't already exist.

Associated forum post URL

No response

rclone version

rclone v1.74.3

• os/version: ubuntu 25.10 (64 bit)
• os/kernel: 6.17.0-40-generic (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.26.4
• go/linking: static
• go/tags: none

What problem are you trying to solve?

When syncing from a bucket that is being written to at the same time, rclone often fails the whole job because a source object is deleted between the listing phase and the transfer phase.

rclone lists the source and queues an object for transfer. Before the transfer runs, the object is deleted at the source. When rclone opens it to copy it, the source returns a 404:

failed to open source object: googleapi: got HTTP response code 404 with body: No such object: my-bucket/path/to/file.jpg

This 404 is counted as an error. Two things follow from that:

1. rclone refuses to delete on the destination:
   not deleting files as there were IO errors
   not deleting directories as there were IO errors
2. The sync returns a non-zero result, so the job is reported as failed.

This happens on every run against live buckets. In our case it is GCS to an S3 backend, scheduled daily, and around 5% of runs fail this way every week. The data is fine. The only problem is that an object listed earlier no longer exists at transfer time, which is normal for a bucket that keeps changing.

The existing flags do not solve this:

• --ignore-errors: lets the deletion happen, but the job still returns a non-zero result, because the error stays counted.
• --retries: re-runs the whole sync. On a bucket that keeps changing, each retry hits new deleted objects, so it does not converge. It also re-lists the whole bucket every attempt, which is expensive on buckets with millions of objects.
• --low-level-retries: a 404 is not retryable, so this does nothing here.
• --min-age: reduces the window but does not remove it, since an old object can also be deleted during the run.

This was discussed on the forum and the conclusion at the time was that there is no flag for it and the workaround is to parse the log: https://forum.rclone.org/t/how-to-ignore-failed-to-copy-failed-to-open-source-object-errors/21734

How do you think rclone should be changed to solve that?

Add a new flag, off by default, for example --skip-missing-source.

When the flag is set and opening a source object fails because the object no longer exists, rclone would:

• log it as a skipped file, not an error
• not count it as an error, so destination deletions still run and the job returns success
• leave all other errors untreated, so real failures (auth, destination down, listing errors) still block deletions and fail the job as they do today

With the flag off, behaviour stays exactly as it is now.

Implementation outline, for discussion:

• The source 404 reaches fs/operations/copy.go as a backend specific error (a *googleapi.Error for GCS, an S3 error for S3), not as fs.ErrorObjectNotFound. So matching on the open error directly is not portable.
• A portable check is to re-stat the source after the open fails: call NewObject on the source remote. Backends translate a missing object to fs.ErrorObjectNotFound, so this works the same across GCS, S3 and others.
• If the re-stat says the object is gone and the flag is set, skip the file without counting an error. Otherwise behave as today.

This stays inside fs/ (config, operations/copy, sync) with no backend changes and no new dependency.

Getting involved

• I'm willing to help implement, test or fund this feature.

Start in fs/operations/copy.go and trace the sync path in fs/ to see how source-open failures are recorded and how configuration flags are defined. The change is done when an opt-in missing-source flag skips objects that disappear during transfer without counting them as errors, while other failures retain current behavior and destination deletions still run.