borgbackup / borgbackup/borg

Add `benchmark crud-modes`

Aperta
#9,167 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
13.7k
Fork
875
Merge medio
12h 37m
PR unite (30g)
199

Descrizione

/kind enhancement

Related: #9165 and #9166

Following the discussion in #9104 it's quite hard to properly benchmark Borg's various encryption modes right now. First, it's hard to benchmark multiple modes for comparison, because the user is required to manually create and later delete test repos using the designated encryption modes. Second, it's hard to use Borg's benchmark results to draw conclusions, see #9165 and #9166.

I thus like to suggest adding a new `benchmark crud-modes` command (name is open for debate, didn't find a better one…) that is basically a wrapper for `benchmark crud` to create and later delete test repos with different encryption modes. Besides making benchmarking easier, it also promotes actually comparing the performance of different encryption modes, and not deciding based on a feeling what might be best.

This command should be added to both Borg 1.4 and Borg 2. The command should benchmark all available encryption modes after another. AFAIK there's no benefit in benchmarking both `repokey` and `keyfile`, so Borg should just benchmark one. Basically, `benchmark crud-mode` should do the following (assuming Borg 1.4; with Borg 2 the commands and encryption modes are different):

```sh
export BORG_PASSPHRASE=mypassphrase
for e in none authenticated authenticated-blake2 repokey repokey-blake2; do
borg init -e $e ./$e

echo "$e $(printf '%.0s=' $(seq $(( 64 - ${#e} - 1 ))))"
mkdir ./bench_data
borg benchmark crud ./$e ./bench_data

borg delete ./$e
rmdir bench_data
done
```

The results shown by `benchmark crud-modes` are identical to that of `benchmark crud`, with the only exception being that the results are grouped by encryption mode. Additionally, it might be reasonable to also print a summary:

**Including a summary**

I'm not entirely sure yet whether `benchmark crud-modes` should also analyse the results similar to what I did manually in https://github.com/borgbackup/borg/issues/9104#issuecomment-3482694134 and https://github.com/borgbackup/borg/issues/9104#issuecomment-3493049066, but I tend to "yes". I don't necessarily mean drawing conclusions, that's a bit hard to implement, but e.g. calculating and printing averages for various commands and samples. For example, `benchmark crud-modes` could include a summary section that prints the averages over all samples per command and encryption mode. Such summary section could look like the following (using results from #9104 and including just two encryption modes). WDYT?

```
summary ========================================================
C-*-* 466.94 MB/s with none encryption (average of all samples: 1.87s)
R-*-* 376.14 MB/s with none encryption (average of all samples: 1.64s)
U-*-* 4,543.96 MB/s with none encryption (average of all samples: 0.41s)
D-*-* 28,203.73 MB/s with none encryption (average of all samples: 0.05s)
C-*-* 487.28 MB/s with authenticated-blake2 encryption (average of all samples: 1.67s)
R-*-* 495.45 MB/s with authenticated-blake2 encryption (average of all samples: 1.22s)
U-*-* 3,672.36 MB/s with authenticated-blake2 encryption (average of all samples: 0.44s)
D-*-* 10,657.19 MB/s with authenticated-blake2 encryption (average of all samples: 0.10s)
```

**Support of `--json-lines`**

Similar to the suggested `benchmark crud --json-lines` (see #9165), the new `benchmark crud-modes` command should also support the `--json-lines` option that yields the same output as `benchmark crud --json-lines`, but additionally includes an `encryption` key and includes the mode used in its `id` key. Here's an example line with `--json-lines`:

```json
{"id": "authenticated_C-Z-BIG", "encryption": "authenticated", "command": "create1", "sample": "Z-BIG", "sample_count": 10, "sample_size": 100000000, "sample_random": false, "time": 4.460000, "io": 224320000}
```

With `--json-lines` the summary section should be omitted.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Parti dal comando esistente `benchmark crud` e dalle proposte correlate in #9165 e #9166, quindi esamina la discussione in #9104. Definisci il comportamento del comando per Borg 1.4 e Borg 2, inclusi il raggruppamento per modalità di crittografia, i riepiloghi opzionali e `--json-lines`; il lavoro è completo quando tutte le modalità supportate sono state sottoposte a benchmark e l'output corrisponde al formato indicato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
cli, performance
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.