rclone / rclone/rclone

Create a --retries-failed-only option

Open
#2,291 7 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bandwidth change detection enhancement
Dominant language
Go
Stars
59.8k
Forks
5.4k
Avg merge
2d 14h
Merged PRs (30d)
48

Description

As stated in the documentation, the --retries int option "Retries the entire sync", which requires a lot of useless work on both sides for just few files having failed on a very big tree.

Would it be possible to create a --retries-failed-only int option ?

I imagine it working like this :

  • each time a file transfer fails, add its source and destination paths in a list kept in memory
  • at the end of the sync, retry only the content of this list instead of retrying the entire sync
    Of course this list could be limited to, for example, 100 failing files. If the list reaches this limit, then a full sync is triggered instead.

In a sync I am currently running, I see 10 or 20 errors for more than 100.000 files being processed, which means that the next retry will check all these files again whereas it should, in my opinion, only retry the failed ones.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the documentation for the existing --retries option and tracing its implementation through sync failure handling. Define how failed source and destination paths are retained, how the 100-file limit triggers a full sync, and verify that retries process only recorded failures when the limit is not reached.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.