RustCrypto / RustCrypto/hashes

blake2 simd benches produce unexpected results

Open
#407 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
2.3k
Forks
340
Avg merge
12h 31m
Merged PRs (30d)
2

Description

I ran blake2 tests with and without simd feature enabled and found that behavior is strange and doesn't make a lot of sense.

Default (no simd):

$ cargo +nightly bench
...
running 8 tests
test blake2b512_10    ... bench:           8 ns/iter (+/- 0) = 1250 MB/s
test blake2b512_100   ... bench:          74 ns/iter (+/- 4) = 1351 MB/s
test blake2b512_1000  ... bench:         708 ns/iter (+/- 85) = 1412 MB/s
test blake2b512_10000 ... bench:       7,005 ns/iter (+/- 531) = 1427 MB/s
test blake2s256_10    ... bench:          13 ns/iter (+/- 0) = 769 MB/s
test blake2s256_100   ... bench:         121 ns/iter (+/- 7) = 826 MB/s
test blake2s256_1000  ... bench:       1,155 ns/iter (+/- 62) = 865 MB/s
test blake2s256_10000 ... bench:      11,791 ns/iter (+/- 1,005) = 848 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 28.22s

Simd:

$ cargo +nightly bench --features simd
...
running 8 tests
test blake2b512_10    ... bench:          11 ns/iter (+/- 0) = 909 MB/s
test blake2b512_100   ... bench:         102 ns/iter (+/- 0) = 980 MB/s
test blake2b512_1000  ... bench:       1,010 ns/iter (+/- 11) = 990 MB/s
test blake2b512_10000 ... bench:      10,076 ns/iter (+/- 68) = 992 MB/s
test blake2s256_10    ... bench:          16 ns/iter (+/- 0) = 625 MB/s
test blake2s256_100   ... bench:         152 ns/iter (+/- 1) = 657 MB/s
test blake2s256_1000  ... bench:       1,483 ns/iter (+/- 12) = 674 MB/s
test blake2s256_10000 ... bench:      14,750 ns/iter (+/- 344) = 677 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 2.43s

I expected simd version to produce better results and judging by the total execution time it is the case, but throughput reported by the benchmark says the opposite.

I suspect something is wrong with benchmarking code.

Using latest master (cc523733e5ba84b5a366825a7d1150bbbfea368b right now) and rustc 1.65.0-nightly (20ffea693 2022-08-11).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the two cargo +nightly bench commands shown for the default and simd configurations, then inspect the Blake2 benchmark entry points and how throughput is calculated. Done means the reported MB/s values accurately reflect the measured benchmark times and remain consistent between the two configurations.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cryptography, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.