Add Bandwidth limiters to backups
- Dominant language
- Shell
- Stars
- 538
- Forks
- 69
- Avg merge
- 3h 28m
- Merged PRs (30d)
- 2
Description
I noticed in my Oracle Cloud VPS setup when the backup is ran it sometimes blows up my network latency on the server for a short period of time it'll go from 50ms to up to 300ms for a bit and even disconnect some far away bedrock players because of this spike.
I'm using rclone/restic and found some command to help limit the backup network bandwidth
Restic:
https://restic.readthedocs.io/en/latest/manual_rest.html#usage-help
```
--limit-download rate limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload rate limits uploads to a maximum rate in KiB/s. (default: unlimited)
```
Rclone:
https://rclone.org/docs/#bwlimit-bandwidth-spec
`--bwlimit=BANDWIDTH_SPEC`
> This option controls the bandwidth limit. For example
`--bwlimit 10M`
> would mean limit the upload and download bandwidth to 10 MiB/s. NB this is bytes per second not bits per second. To use a single limit, specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P. The default is 0 which means to not limit bandwidth.
I'm playing with the backup-loop.sh script right now to get this working so far changed the following bits of code
(this would not be the default values, i'd change them to the unlimited default values and then use a environment variable in the docker compose/dockerfile to set it.)
```bash
: "${RESTIC_BW_DL:=20480}"
: "${RESTIC_BW_UP:=5120}"
: "${RCLONE_BW_LIMIT:=10m:20m}"
```
```bash
if ! command rclone copy "${outFile}" "${RCLONE_REMOTE}:${RCLONE_DEST_DIR}" --bwlimit "${RCLONE_BW_LIMIT}" ; then
```
```bash
backup() {
log INFO "Backing up content in ${SRC_DIR} as host ${RESTIC_HOSTNAME}"
args=(
--host "${RESTIC_HOSTNAME}"
--limit-download "${RESTIC_BW_DL}" #change made
--limit-upload "${RESTIC_BW_UP}" #change made
)
if isDebug || isTrue "$RESTIC_VERBOSE"; then
args+=(-vv)
fi
(cd "$SRC_DIR" &&
command restic backup "${args[@]}" "${restic_tags_arguments[@]}" "${excludes[@]}" "${includes_patterns[@]}" | log INFO
)
}
```
Just starting to test this but figured I'd put this in as and Idea and may be other backup methods that have this argument as well that may have a similar feature
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with backup-loop.sh and trace how the restic and rclone commands are assembled, then inspect the Docker Compose/Dockerfile environment configuration mentioned in the issue. Confirm how bandwidth settings should be passed for each backup method and ensure the completed behavior can be configured without changing the script.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, shell
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100