intel / intel/confidential-computing.sgx.sdk

Very Poor Multi-Threaded Performance of lib TCrypto Hashing Functions

Open
#168 8 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
2
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Hello,

We've been investigating performance issues when using the hashing function provided by the tcrypto library.
We have isolated this issue to the way the IPP cryptography primitives are being used: https://github.com/intel/cryptography-primitives/issues/93

To take the specific example of SHA-256, when calling `ippsHashMessage_rmf`, `ippsHashMethod_SHA256_TT` is called every time:

https://github.com/intel/linux-sgx/blob/7385e10ce1106215d15f874a024ca224c7417eea/sdk/tlibcrypto/ipp/sgx_sha256_msg.cpp#L49-L68

This seems innocent enough as this code should just return a static structure with function pointers to the specific functions for that hashing primitive. However, the `_TT` methods support dynamic dispatching to the NI implementations of those hashing primitives: https://www.intel.com/content/www/us/en/docs/ipp-crypto/developer-guide-reference/2021-9/one-way-hash-primitives.html

Because of the way this was implemented, calling `ippsHashMethod_SHA256_TT` repeatedly, on a platform supporting SHA-NI, results in every call setting the `method.hashUpdate` global function pointer to the normal implementation function pointer and then to the NI one:

https://github.com/intel/cryptography-primitives/blob/59a3c2e80c8fccd0d37b7a58020671c5468ec49b/sources/ippcp/hash/sha256/pcphashmethod_sha256_tt.c#L49-L75

Since this structure is static and shared across all threads, calling this method from different threads causes the function pointer to keep changing for all threads involved with devastating consequences for the memory caches (on the pure ippcp sample without using SGX, we could see a massive memory bottleneck due to L1D and L3 cache misses using perf).

I'm unsure what is the correct fix for the libtcrypto functions. I have seen some internal code implementing CPU dispatching directly using ippcp internal functions: https://github.com/intel/linux-sgx/blob/7385e10ce1106215d15f874a024ca224c7417eea/sdk/tlibcrypto/ipp/ipp_disp/intel64/ippsHashMessage_rmf.c#L61-L76

For our use case, we will use ippcp directly, make sure to call `ippsHashMethod_SHA256_TT` only once and cache it.

Contributor guide

Open the contributing guide

Research direction

Start with sdk/tlibcrypto/ipp/sgx_sha256_msg.cpp and the dispatch implementation in sdk/tlibcrypto/ipp/ipp_disp/intel64/ippsHashMessage_rmf.c. Compare their handling of ippsHashMethod_SHA256_TT with the linked cryptography-primitives implementation, then reproduce the multithreaded hashing behavior. Done means the affected libtcrypto hashing path no longer causes shared dispatch state to degrade concurrent performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cryptography, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.