HTTP remote: add an option to forward write operations to another remote
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 59.8k
- Forks
- 5.4k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 48
Description
What is your current rclone version (output from rclone version)?
Not the only system I use it on, but this is what I'm creating the issue from:
rclone v1.71.1
- os/version: Microsoft Windows 11 Home 24H2 24H2 (64 bit)
- os/kernel: 10.0.26100.6584 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.25.1
- go/linking: static
- go/tags: cmount
What problem are you are trying to solve?
I'm using a rented server for temporary file storage before I get home and move stuff onto my local NAS. I've found that using an HTTP remote for the server's included file lists is the most reliable way to download them, but I do get a bit annoyed at having to then manually delete each file from source using another method after the downloads complete successfully.
How do you think rclone should be changed to solve that?
I was thinking that since the HTTP storage core is read-only, we could provide a way of linking an HTTP remote to a different write-capable remote for accessing the same server through a different protocol such as SFTP. As an example of what this could look like in the config file:
[something]
type = http
url = https://foo:bar@something.com/files/
write_remote = /:_something_sftp:/var/www/files/ # destination_path:remote:matching_remote_path
And so, any write operations on the HTTP remote (move files on remote, delete from source after moving to destination, etc.) would happen over that other SFTP remote. Unless they're on a path that isn't covered by the write_remote option, in which case they'd still throw the usual read-only error.
How to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the HTTP storage core and its read-only operation handling, then review how the proposed SFTP write remote and path mapping would fit. Done means write operations covered by write_remote use the alternate remote while uncovered paths retain the usual read-only error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100