Use interleaved SHA-NI for SHA-256 on some CPUs lacking AVX-512
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 13.6k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Description
In https://github.com/openwall/john/issues/5435#issuecomment-1943397943 @ukasz wrote:
I wanted to play with sha256 specifically, because I noticed some improvement
in the latency of sha-ni instructions on newer architectures (compared to the initial relase) on Agner's fog website.
I was wondering how that could stack against john's implementation.
Some initial testing showed that due to low register usage on sha256 we should be able to calculate two hashes at once using sha extensions and this results with 1.5x perf compared to calculating just one hash.
So in general the question was what is faster 1.5x with sha-ni or john's avx implementation, unfortunately I don't know yet.
As it happens, @alainesp was also experimenting with that just recently:
https://github.com/alainesp/fast-small-crypto
The preliminary results we have suggest that on some AMD CPUs, 2x interleaved SHA-NI can be almost twice faster than 1x, and ~75% faster than AVX2: https://github.com/alainesp/fast-small-crypto/actions/runs/7876924916/job/21491982542 (I only guess that this ran on an AMD CPU, but apparently it's similar to Alain's testing on his known AMD).
However, in my testing of Alain's code on Intel Tiger Lake (11th gen) and building with gcc 11, 1x and 2x SHA-NI are similar speed to each other, and are very slightly slower than AVX2, and almost 3 times slower than AVX-512.
There's no improvement from SHA-NI for SHA-1 anywhere we tested.
@ukasz What CPUs did you see improved latencies for, and what CPU are you testing on? Maybe things improved on newer Intel CPUs. If any of those lack AVX-512, it could be reasonable to use SHA-NI there as well.
I wonder if it would make sense to mix SHA-NI and AVX2 or AVX-512 instructions on any CPUs. I guess this depends on what execution ports these groups of instructions utilize.
Separately, I hear similar instructions for SHA-512 are coming in near future CPUs. I guess those will outperform AVX2, but not necessarily outperform AVX-512.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is named. Start with the linked issue discussion and Alain Espinosa's fast-small-crypto results, then benchmark interleaved SHA-NI against AVX2 and AVX-512 on the relevant CPUs; done would require a benchmark-backed implementation decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- cryptography, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100