borgbackup / borgbackup/borg

Add `benchmark crud-modes`

Abierto
#9,167 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
13.7k
Forks
875
Merge medio
13 h 24 min
PR fusionados (30 d)
183

Descripción

/kind enhancement

Related: #9165 and #9166

Following the discussion in #9104 it's quite hard to properly benchmark Borg's various encryption modes right now. First, it's hard to benchmark multiple modes for comparison, because the user is required to manually create and later delete test repos using the designated encryption modes. Second, it's hard to use Borg's benchmark results to draw conclusions, see #9165 and #9166.

I thus like to suggest adding a new `benchmark crud-modes` command (name is open for debate, didn't find a better one…) that is basically a wrapper for `benchmark crud` to create and later delete test repos with different encryption modes. Besides making benchmarking easier, it also promotes actually comparing the performance of different encryption modes, and not deciding based on a feeling what might be best.

This command should be added to both Borg 1.4 and Borg 2. The command should benchmark all available encryption modes after another. AFAIK there's no benefit in benchmarking both `repokey` and `keyfile`, so Borg should just benchmark one. Basically, `benchmark crud-mode` should do the following (assuming Borg 1.4; with Borg 2 the commands and encryption modes are different):

```sh
export BORG_PASSPHRASE=mypassphrase
for e in none authenticated authenticated-blake2 repokey repokey-blake2; do
borg init -e $e ./$e

echo "$e $(printf '%.0s=' $(seq $(( 64 - ${#e} - 1 ))))"
mkdir ./bench_data
borg benchmark crud ./$e ./bench_data

borg delete ./$e
rmdir bench_data
done
```

The results shown by `benchmark crud-modes` are identical to that of `benchmark crud`, with the only exception being that the results are grouped by encryption mode. Additionally, it might be reasonable to also print a summary:

**Including a summary**

I'm not entirely sure yet whether `benchmark crud-modes` should also analyse the results similar to what I did manually in https://github.com/borgbackup/borg/issues/9104#issuecomment-3482694134 and https://github.com/borgbackup/borg/issues/9104#issuecomment-3493049066, but I tend to "yes". I don't necessarily mean drawing conclusions, that's a bit hard to implement, but e.g. calculating and printing averages for various commands and samples. For example, `benchmark crud-modes` could include a summary section that prints the averages over all samples per command and encryption mode. Such summary section could look like the following (using results from #9104 and including just two encryption modes). WDYT?

```
summary ========================================================
C-*-* 466.94 MB/s with none encryption (average of all samples: 1.87s)
R-*-* 376.14 MB/s with none encryption (average of all samples: 1.64s)
U-*-* 4,543.96 MB/s with none encryption (average of all samples: 0.41s)
D-*-* 28,203.73 MB/s with none encryption (average of all samples: 0.05s)
C-*-* 487.28 MB/s with authenticated-blake2 encryption (average of all samples: 1.67s)
R-*-* 495.45 MB/s with authenticated-blake2 encryption (average of all samples: 1.22s)
U-*-* 3,672.36 MB/s with authenticated-blake2 encryption (average of all samples: 0.44s)
D-*-* 10,657.19 MB/s with authenticated-blake2 encryption (average of all samples: 0.10s)
```

**Support of `--json-lines`**

Similar to the suggested `benchmark crud --json-lines` (see #9165), the new `benchmark crud-modes` command should also support the `--json-lines` option that yields the same output as `benchmark crud --json-lines`, but additionally includes an `encryption` key and includes the mode used in its `id` key. Here's an example line with `--json-lines`:

```json
{"id": "authenticated_C-Z-BIG", "encryption": "authenticated", "command": "create1", "sample": "Z-BIG", "sample_count": 10, "sample_size": 100000000, "sample_random": false, "time": 4.460000, "io": 224320000}
```

With `--json-lines` the summary section should be omitted.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Parte del comando existente `benchmark crud` y de las propuestas relacionadas en #9165 y #9166, y después revisa la discusión en #9104. Define el comportamiento del comando para Borg 1.4 y Borg 2, incluida la agrupación por modos de cifrado, los resúmenes opcionales y `--json-lines`; se considera terminado cuando se hayan sometido a benchmark todos los modos compatibles y la salida coincida con el formato indicado.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
cli, performance
Tipo de issue
Nueva funcionalidad
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
42/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.