Add `benchmark crud-modes`
- 主要言語
- Python
- スター
- 13.7k
- フォーク
- 875
- 平均マージ
- 13時間 24分
- マージ済み PR(30日)
- 183
説明
/kind enhancement
Related: #9165 and #9166
Following the discussion in #9104 it's quite hard to properly benchmark Borg's various encryption modes right now. First, it's hard to benchmark multiple modes for comparison, because the user is required to manually create and later delete test repos using the designated encryption modes. Second, it's hard to use Borg's benchmark results to draw conclusions, see #9165 and #9166.
I thus like to suggest adding a new `benchmark crud-modes` command (name is open for debate, didn't find a better one…) that is basically a wrapper for `benchmark crud` to create and later delete test repos with different encryption modes. Besides making benchmarking easier, it also promotes actually comparing the performance of different encryption modes, and not deciding based on a feeling what might be best.
This command should be added to both Borg 1.4 and Borg 2. The command should benchmark all available encryption modes after another. AFAIK there's no benefit in benchmarking both `repokey` and `keyfile`, so Borg should just benchmark one. Basically, `benchmark crud-mode` should do the following (assuming Borg 1.4; with Borg 2 the commands and encryption modes are different):
```sh
export BORG_PASSPHRASE=mypassphrase
for e in none authenticated authenticated-blake2 repokey repokey-blake2; do
borg init -e $e ./$e
echo "$e $(printf '%.0s=' $(seq $(( 64 - ${#e} - 1 ))))"
mkdir ./bench_data
borg benchmark crud ./$e ./bench_data
borg delete ./$e
rmdir bench_data
done
```
The results shown by `benchmark crud-modes` are identical to that of `benchmark crud`, with the only exception being that the results are grouped by encryption mode. Additionally, it might be reasonable to also print a summary:
**Including a summary**
I'm not entirely sure yet whether `benchmark crud-modes` should also analyse the results similar to what I did manually in https://github.com/borgbackup/borg/issues/9104#issuecomment-3482694134 and https://github.com/borgbackup/borg/issues/9104#issuecomment-3493049066, but I tend to "yes". I don't necessarily mean drawing conclusions, that's a bit hard to implement, but e.g. calculating and printing averages for various commands and samples. For example, `benchmark crud-modes` could include a summary section that prints the averages over all samples per command and encryption mode. Such summary section could look like the following (using results from #9104 and including just two encryption modes). WDYT?
```
summary ========================================================
C-*-* 466.94 MB/s with none encryption (average of all samples: 1.87s)
R-*-* 376.14 MB/s with none encryption (average of all samples: 1.64s)
U-*-* 4,543.96 MB/s with none encryption (average of all samples: 0.41s)
D-*-* 28,203.73 MB/s with none encryption (average of all samples: 0.05s)
C-*-* 487.28 MB/s with authenticated-blake2 encryption (average of all samples: 1.67s)
R-*-* 495.45 MB/s with authenticated-blake2 encryption (average of all samples: 1.22s)
U-*-* 3,672.36 MB/s with authenticated-blake2 encryption (average of all samples: 0.44s)
D-*-* 10,657.19 MB/s with authenticated-blake2 encryption (average of all samples: 0.10s)
```
**Support of `--json-lines`**
Similar to the suggested `benchmark crud --json-lines` (see #9165), the new `benchmark crud-modes` command should also support the `--json-lines` option that yields the same output as `benchmark crud --json-lines`, but additionally includes an `encryption` key and includes the mode used in its `id` key. Here's an example line with `--json-lines`:
```json
{"id": "authenticated_C-Z-BIG", "encryption": "authenticated", "command": "create1", "sample": "Z-BIG", "sample_count": 10, "sample_size": 100000000, "sample_random": false, "time": 4.460000, "io": 224320000}
```
With `--json-lines` the summary section should be omitted.
コントリビューションガイド
調査の方向性
既存の `benchmark crud` コマンドと #9165 および #9166 の関連提案から始め、その後 #9104 の議論を確認します。暗号化モードによるグループ化、オプションのサマリー、`--json-lines` を含め、Borg 1.4 と Borg 2 におけるコマンドの動作を定義します。対応するすべてのモードでベンチマークを実行し、出力が指定された形式と一致すれば完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- cli, performance
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 42/100