borgbackup / borgbackup/borg

Add `benchmark crud-modes`

オープン
#9,167 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
13.7k
フォーク
875
平均マージ
13時間 24分
マージ済み PR(30日)
183

説明

/kind enhancement

Related: #9165 and #9166

Following the discussion in #9104 it's quite hard to properly benchmark Borg's various encryption modes right now. First, it's hard to benchmark multiple modes for comparison, because the user is required to manually create and later delete test repos using the designated encryption modes. Second, it's hard to use Borg's benchmark results to draw conclusions, see #9165 and #9166.

I thus like to suggest adding a new `benchmark crud-modes` command (name is open for debate, didn't find a better one…) that is basically a wrapper for `benchmark crud` to create and later delete test repos with different encryption modes. Besides making benchmarking easier, it also promotes actually comparing the performance of different encryption modes, and not deciding based on a feeling what might be best.

This command should be added to both Borg 1.4 and Borg 2. The command should benchmark all available encryption modes after another. AFAIK there's no benefit in benchmarking both `repokey` and `keyfile`, so Borg should just benchmark one. Basically, `benchmark crud-mode` should do the following (assuming Borg 1.4; with Borg 2 the commands and encryption modes are different):

```sh
export BORG_PASSPHRASE=mypassphrase
for e in none authenticated authenticated-blake2 repokey repokey-blake2; do
borg init -e $e ./$e

echo "$e $(printf '%.0s=' $(seq $(( 64 - ${#e} - 1 ))))"
mkdir ./bench_data
borg benchmark crud ./$e ./bench_data

borg delete ./$e
rmdir bench_data
done
```

The results shown by `benchmark crud-modes` are identical to that of `benchmark crud`, with the only exception being that the results are grouped by encryption mode. Additionally, it might be reasonable to also print a summary:

**Including a summary**

I'm not entirely sure yet whether `benchmark crud-modes` should also analyse the results similar to what I did manually in https://github.com/borgbackup/borg/issues/9104#issuecomment-3482694134 and https://github.com/borgbackup/borg/issues/9104#issuecomment-3493049066, but I tend to "yes". I don't necessarily mean drawing conclusions, that's a bit hard to implement, but e.g. calculating and printing averages for various commands and samples. For example, `benchmark crud-modes` could include a summary section that prints the averages over all samples per command and encryption mode. Such summary section could look like the following (using results from #9104 and including just two encryption modes). WDYT?

```
summary ========================================================
C-*-* 466.94 MB/s with none encryption (average of all samples: 1.87s)
R-*-* 376.14 MB/s with none encryption (average of all samples: 1.64s)
U-*-* 4,543.96 MB/s with none encryption (average of all samples: 0.41s)
D-*-* 28,203.73 MB/s with none encryption (average of all samples: 0.05s)
C-*-* 487.28 MB/s with authenticated-blake2 encryption (average of all samples: 1.67s)
R-*-* 495.45 MB/s with authenticated-blake2 encryption (average of all samples: 1.22s)
U-*-* 3,672.36 MB/s with authenticated-blake2 encryption (average of all samples: 0.44s)
D-*-* 10,657.19 MB/s with authenticated-blake2 encryption (average of all samples: 0.10s)
```

**Support of `--json-lines`**

Similar to the suggested `benchmark crud --json-lines` (see #9165), the new `benchmark crud-modes` command should also support the `--json-lines` option that yields the same output as `benchmark crud --json-lines`, but additionally includes an `encryption` key and includes the mode used in its `id` key. Here's an example line with `--json-lines`:

```json
{"id": "authenticated_C-Z-BIG", "encryption": "authenticated", "command": "create1", "sample": "Z-BIG", "sample_count": 10, "sample_size": 100000000, "sample_random": false, "time": 4.460000, "io": 224320000}
```

With `--json-lines` the summary section should be omitted.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

既存の `benchmark crud` コマンドと #9165 および #9166 の関連提案から始め、その後 #9104 の議論を確認します。暗号化モードによるグループ化、オプションのサマリー、`--json-lines` を含め、Borg 1.4 と Borg 2 におけるコマンドの動作を定義します。対応するすべてのモードでベンチマークを実行し、出力が指定された形式と一致すれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
cli, performance
issue の種類
機能追加
難易度
5/5
見積もり時間
1週間以上
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
42/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。