hmac_digest race condition with openssl 3.0.19 and Python 3.14t

Crash report

What happened?

summary

I'm seeing a null pointer deref happen often in this stack, after hmac.digest(...) calls in a thread in my app during early startup. I have python 3.14t and openssl 3.0.19 LTS (with python's internal _hmac backend disabled because I can't statically link it). The crash does not happen if I use hmac.new(...).digest(), which takes a slightly different path through openssl. I will note python's _ssl module may also be able to trigger this.

This is happening in a complex app, I haven't been able to find an easy standalone repro.

0   libsystem_pthread.dylib       	0x00007fff2037b714 pthread_rwlock_rdlock + 1
1   libcrypto.3.dylib             	0x00000001022a90f8 ossl_lib_ctx_get_data + 72
2   libcrypto.3.dylib             	0x000000010227b6ce 0x102174000 + 1078990
3   libcrypto.3.dylib             	0x000000010227b089 0x102174000 + 1077385
4   libcrypto.3.dylib             	0x000000010227b02a evp_generic_fetch + 106
5   libcrypto.3.dylib             	0x000000010228c6b3 EVP_MAC_fetch + 67
6   libcrypto.3.dylib             	0x000000010228c363 EVP_Q_mac + 99
7   libcrypto.3.dylib             	0x00000001022a235d HMAC + 253

I have also tested with openssl 3.5.x LTS, and the default libctx issue appears to be fixed in openssl there as they've reworked the default libctx init (they have more explicit init guards).

I don't know what to actually do about this. Maybe the solution is a build-time check against pairing free-threaded Python with pre-3.5 openssl? I also haven't checked the other compatible ssl impls (e.g. boringssl and libressl) to see if they have the same threading issues or if they've fixed them independently.

more detail

If you're using openssl as your hmac backend, _hashopenssl.hmac_digest() can race something inside openssl and crash if you have threads doing hmac during early Python startup. I think this is due to hmac_digest using the implicit default_context_int libctx inside openssl, which doesn't seem thread safe on startup in 3.0.19.
At _hashopenssl.c:66, we see PY_EVP_MD_fetch is defined to use EVP_MD_fetch(NULL, ...), which selects the default libctx.

If there are other parts of cpython using the implicit openssl libctx rather than creating their own context, I suspect those are also not safe with openssl 3.0.19. It looks like the next openssl LTS (3.5.x) may have a better locking mechanism around the default libctx.

It looks like _ssl also uses the default openssl libctx: _ssl.c:3449 - SSL_CTX_new(method) implies the default libctx (where SSL_new_ex() allows you to specify libctx)

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

No response

從 Modules/_hashopenssl.c 第 66 行附近開始，追蹤在線程化早期啟動期間 _hashopenssl.hmac_digest() 如何到達 OpenSSL 的預設 libctx。將該路徑與 hmac.new(...).digest() 進行比較，並檢查 _ssl.c 第 3449 行附近是否有類似的預設 libctx 使用。對於受影響的 OpenSSL 版本和相容的 SSL 實作，確立可重現的失敗，或確認一項安全的處理策略，即視為完成。