repo-info --stats: report repository size (and optionally dedup stats) read-only, with --json
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 875
- Avg merge
- 11h 15m
- Merged PRs (30d)
- 192
Description
## Problem
Borg 2 has no read-only way to learn how large a repository is.
- `repo-info --json` reports id, location, `last_modified`, encryption and cache paths, no size.
- `info --json` (archive level) reports `nfiles` and `original_size` only. Its help says: "for the repository-wide deduplicated size, use borg compact --stats."
- `compact --stats` does report it ("Repository size is 402 kB in 8 objects", plus source data size, deduplicated size, compression factor), but only while compacting: it needs the exclusive lock, rewrites packs, and `compact --dry-run --stats` answers "Ignoring --stats. It is not supported when using --dry-run." The numbers are also only printed as rounded INFO log lines, so a program has to parse "402 kB" out of a `log_message`.
- `repo-list --format "{size}"` is per-archive original size, not repository size.
For anything that wants to show or monitor repository size without changing the repository (a UI, a Prometheus exporter, a cron report), the only options today are store-level measurements (`du`, `rclone size`, walking the borgstore backend), which measure files, not Borg objects, and which cannot be expressed uniformly across `file:`, `sftp:`, `rest:`, `rclone:` and `s3:` stores.
This is the read-only half of #8796 (closed with "use `compact --stats -v`") and overlaps the monitoring wish in #2934.
## Observation: the data is already there and cheap
Measured on 2.0.0b23 and 2.0.0b24 (identical):
1. **Repository size** needs nothing but the chunk index. `Repository.list()` yields `(chunk_id, storage_size)` from the `index/` fragments under a shared lock, without loading a pack. Summing it reproduces `compact --stats`'s "Repository size" byte for byte (402434 bytes = "402 kB in 8 objects"; `du` of `packs/` is ~5 % higher through pack headers). On a small scratch repository this takes 15 ms.
```python
from borg.logger import setup_logging; setup_logging()
from borg.repository import Repository
from borg.helpers import Location
with Repository(Location(url), exclusive=False) as r:
sizes = [s for _, s in r.list()]
print(len(sizes), sum(sizes))
```
2. **Source data size, file count and deduplicated size** are what `CompactCmd.analyze_archives()` computes, and since the archive reference caches landed (`cache/referenced_by_archive_` in the store, `get_archive_references(..., cached=True)`), an unchanged archive is not even opened again. A read-only variant is exactly the `dry_run` path of that method (`store=False`, no cache cleanup).
So a read-only `repo-info --stats` is a re-use of two existing code paths, not new bookkeeping.
## Proposal
Add `--stats` to `repo-info`:
```
borg repo-info --stats [--json]
```
Text output (same wording as `compact --stats`, so users recognise it):
```
Repository size: 402 kB in 8 objects
Source data size: 700 kB in 2 files (2 archives)
Deduplicated size: 400 kB
Deduplication factor: 0.57
Compression factor: 1.00
```
JSON: add a `stats` object with exact byte values, alongside the existing keys, so `repo-info --json` stays backwards compatible without the flag (values illustrative):
```json
{
"repository": {"id": "...", "location": "...", "last_modified": "..."},
"encryption": {...},
"cache": {...},
"stats": {
"repository_size": 402434,
"object_count": 8,
"source_size": 700436,
"source_files": 2,
"archive_count": 2,
"deduplicated_size": 400304
}
}
```
Semantics:
- `repository_size` / `object_count`: sum and count over the chunk index (what `compact --stats` prints after compaction). Always available, shared lock, no pack access.
- `source_size`, `source_files`, `archive_count`, `deduplicated_size`: from the read-only analysis of all archives, using the per-archive reference caches when present. Archives without a cache are scanned (as `compact --dry-run` does today) but nothing is written to the store.
- No `compaction_saved`, since nothing is compacted.
- Requires a shared lock only, so it can run while a backup is in progress, unlike `compact`.
If the archive analysis is considered too expensive for `repo-info`, a smaller first step would already help most callers: `repository_size` and `object_count` only (index sum), with the dedup fields behind a second flag (`--stats --analyze`) or left to `compact --stats`. Alternatively, make `compact --dry-run --stats` honour `--stats` instead of ignoring it; that gives the same numbers read-only, but still under the exclusive lock and without JSON.
## Why not parse `compact --stats -v`
- It changes the repository, so a monitor cannot call it on its own schedule.
- It takes the exclusive lock.
- The values are rounded to three significant digits and only reachable through INFO `log_message` lines.
- Borg 1's `info --json` exposed `cache.stats` with exact `unique_csize`, `unique_size`, `total_size`; tools built on that lost the field with Borg 2 and currently have no replacement.
## Environment
borg 2.0.0b23 and 2.0.0b24, borgstore 0.6.1, Python 3.12, Linux; measurements above on a `file:` store; the same `repo-info` runs against `rest:` stores in production.
Contributor guide
Research direction
Start with the repo-info command and Repository.list() for the shared-lock repository size and object count. Then read CompactCmd.analyze_archives() and its dry_run path to understand the optional archive analysis and reference caches. Done means read-only text and JSON stats with exact values, while preserving existing repo-info JSON output without --stats.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100