borg2: Re-add UNIX socket communication
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 875
- Avg merge
- 11h 15m
- Merged PRs (30d)
- 192
Description
After the removal of the legacy RPC protocol with #9732 (with a few exceptions), support for UNIX socket communication via the `socket:` URI scheme and the `--socket` CLI option (for selecting a non-default socket path) was also removed.
UNIX socket communication was introduced to facilitate a range of different scenarios. It is a rather low-level feature. Low-level features such as this are not necessarily intended to encode specific workflows, but to provide composable building blocks. This allows users to address more niche or complex deployment requirements without Borg itself having to explicitly support or standardize every such scenario. It basically provides a foundation that can be combined with other tooling and system components.
Examples include:
* Local multi-user setups
* Service separation, where one user or service exposes repos and other users connect through a UNIX domain socket
* Privilege separation, allowing repo access through a controlled local endpoint instead of direct filesystem access
* Container-based deployments, where client and server run in separate containers and communicate via a shared socket mount
* Access control via filesystem permissions, utilizing ownership, groups, and file modes on the socket file itself
* systemd socket activation, where `borg serve` is started on demand when a client connects
* More complex remote deployment scenarios
* Using transport mechanisms other than SSH
* Pull-style backup setups
My personal use case is creating full-system backups. In that scenario, `borg create` needs to run as root, while I would prefer the repo itself to be stored in a well-defined location and owned by a dedicated user (e.g. `backup`). Without UNIX socket support, this generally requires either changing ownership with `chown` after Borg operations complete, or using Borg's remote access mechanisms as a local privilege-separation workaround by connecting to `localhost` via SSH as a different user. See the 2023 discussion in #6183 for additional details and my personal motivation.
Thomas assessed in https://github.com/borgbackup/borg/issues/9593#issuecomment-4682854452 that reintroducing UNIX socket communication is not trivial, because Borg currently relies on three communication channels in the SSH-based implementation: stdin, stdout, and stderr. A UNIX domain socket, however, only provides a bidirectional data channel. Currently, the REST protocol uses stdin/stdout for protocol communication and stderr for logging and diagnostic output.
This limitation already existed when `socket:` and `--socket` support was originally introduced in 2023. It was possible to work around it at the time (see https://github.com/borgbackup/borg/issues/6183#issuecomment-1500976094 and #7604 for the 2023 analysis). However, I am not aware of the current situation, i.e. how heavily Borg now relies on stderr (if at all), whether a simple workaround (e.g., temporarily disabling remote logging) would be sufficient, or whether a REST protocol extension (see #9593) would be required to support UNIX sockets again.
**Prior work:**
* Suggestion: #6183 (including discussion about the `--socket` CLI option)
* Pull Request: #7615 (prior: [#7519](https://github.com/borgbackup/borg/pull/7519))
**Related (incomplete list):**
* Pull-style backups with UNIX socket files: #9459 (prior: [#6270](https://github.com/borgbackup/borg/pull/6270)), and https://github.com/borgbackup/borg/blob/master/docs/deployment/pull-backup.rst#socat
* Pull-style backups with fds: #4749
* SSH to `localhost`: https://github.com/borgbackup/borg/issues/3587#issuecomment-443694439 / https://github.com/borgbackup/borg/issues/4082#issuecomment-423679676
* and probably a few more…
Contributor guide
Assessment
This issue has not been assessed yet.