borgbackup / borgbackup/borgstore

sftp: speed / latency / protocol issues

Open
#44 2 comments 0 reactions 0 assignees View on GitHub
documentation wontfix
Dominant language
Python
Stars
25
Forks
8
Avg merge
16m
Merged PRs (30d)
8

Description

Documenting some fundamental issues of the sftp protocol:

- sftp protocol uses 32KB blocks for data transfer and usually each block has to get acknowledged individually, which slows down transfers significantly on connections with notable latency.
- `paramiko` offers pipelining (write) and prefetching (read) to optimize this and `borgstore` uses these features.
- but in the end, `sftp` is just **not** a fast protocol and `paramiko` is not a fast sftp client implementation either.

Thus, using `sftp` is not recommended for big stores nor for high latency connections.

We can't do much about that in `borgstore` besides offering alternatives that use other protocols.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.