itzg / itzg/docker-mc-backup

Add Bandwidth limiters to backups

Open
#209 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Shell
Stars
538
Forks
69
Avg merge
3h 28m
Merged PRs (30d)
2

Description

I noticed in my Oracle Cloud VPS setup when the backup is ran it sometimes blows up my network latency on the server for a short period of time it'll go from 50ms to up to 300ms for a bit and even disconnect some far away bedrock players because of this spike.

I'm using rclone/restic and found some command to help limit the backup network bandwidth

Restic:
https://restic.readthedocs.io/en/latest/manual_rest.html#usage-help

```
--limit-download rate limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload rate limits uploads to a maximum rate in KiB/s. (default: unlimited)
```

Rclone:
https://rclone.org/docs/#bwlimit-bandwidth-spec

`--bwlimit=BANDWIDTH_SPEC`

> This option controls the bandwidth limit. For example

`--bwlimit 10M`

> would mean limit the upload and download bandwidth to 10 MiB/s. NB this is bytes per second not bits per second. To use a single limit, specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P. The default is 0 which means to not limit bandwidth.

I'm playing with the backup-loop.sh script right now to get this working so far changed the following bits of code

(this would not be the default values, i'd change them to the unlimited default values and then use a environment variable in the docker compose/dockerfile to set it.)
```bash
: "${RESTIC_BW_DL:=20480}"
: "${RESTIC_BW_UP:=5120}"
: "${RCLONE_BW_LIMIT:=10m:20m}"
```

```bash
if ! command rclone copy "${outFile}" "${RCLONE_REMOTE}:${RCLONE_DEST_DIR}" --bwlimit "${RCLONE_BW_LIMIT}" ; then
```

```bash
backup() {
log INFO "Backing up content in ${SRC_DIR} as host ${RESTIC_HOSTNAME}"
args=(
--host "${RESTIC_HOSTNAME}"
--limit-download "${RESTIC_BW_DL}" #change made
--limit-upload "${RESTIC_BW_UP}" #change made
)
if isDebug || isTrue "$RESTIC_VERBOSE"; then
args+=(-vv)
fi
(cd "$SRC_DIR" &&
command restic backup "${args[@]}" "${restic_tags_arguments[@]}" "${excludes[@]}" "${includes_patterns[@]}" | log INFO
)
}
```

Just starting to test this but figured I'd put this in as and Idea and may be other backup methods that have this argument as well that may have a similar feature

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with backup-loop.sh and trace how the restic and rclone commands are assembled, then inspect the Docker Compose/Dockerfile environment configuration mentioned in the issue. Confirm how bandwidth settings should be passed for each backup method and ensure the completed behavior can be configured without changing the script.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, shell
Domain
devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.