hmac_digest race condition with openssl 3.0.19 and Python 3.14t

Crash report

What happened?

summary

I'm seeing a null pointer deref happen often in this stack, after hmac.digest(...) calls in a thread in my app during early startup. I have python 3.14t and openssl 3.0.19 LTS (with python's internal _hmac backend disabled because I can't statically link it). The crash does not happen if I use hmac.new(...).digest(), which takes a slightly different path through openssl. I will note python's _ssl module may also be able to trigger this.

This is happening in a complex app, I haven't been able to find an easy standalone repro.

0   libsystem_pthread.dylib       	0x00007fff2037b714 pthread_rwlock_rdlock + 1
1   libcrypto.3.dylib             	0x00000001022a90f8 ossl_lib_ctx_get_data + 72
2   libcrypto.3.dylib             	0x000000010227b6ce 0x102174000 + 1078990
3   libcrypto.3.dylib             	0x000000010227b089 0x102174000 + 1077385
4   libcrypto.3.dylib             	0x000000010227b02a evp_generic_fetch + 106
5   libcrypto.3.dylib             	0x000000010228c6b3 EVP_MAC_fetch + 67
6   libcrypto.3.dylib             	0x000000010228c363 EVP_Q_mac + 99
7   libcrypto.3.dylib             	0x00000001022a235d HMAC + 253

I have also tested with openssl 3.5.x LTS, and the default libctx issue appears to be fixed in openssl there as they've reworked the default libctx init (they have more explicit init guards).

I don't know what to actually do about this. Maybe the solution is a build-time check against pairing free-threaded Python with pre-3.5 openssl? I also haven't checked the other compatible ssl impls (e.g. boringssl and libressl) to see if they have the same threading issues or if they've fixed them independently.

more detail

If you're using openssl as your hmac backend, _hashopenssl.hmac_digest() can race something inside openssl and crash if you have threads doing hmac during early Python startup. I think this is due to hmac_digest using the implicit default_context_int libctx inside openssl, which doesn't seem thread safe on startup in 3.0.19.
At _hashopenssl.c:66, we see PY_EVP_MD_fetch is defined to use EVP_MD_fetch(NULL, ...), which selects the default libctx.

If there are other parts of cpython using the implicit openssl libctx rather than creating their own context, I suspect those are also not safe with openssl 3.0.19. It looks like the next openssl LTS (3.5.x) may have a better locking mechanism around the default libctx.

It looks like _ssl also uses the default openssl libctx: _ssl.c:3449 - SSL_CTX_new(method) implies the default libctx (where SSL_new_ex() allows you to specify libctx)

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

No response

Bắt đầu với Modules/_hashopenssl.c quanh dòng 66 và truy vết cách _hashopenssl.hmac_digest() đi tới libctx mặc định của OpenSSL trong quá trình khởi động sớm có luồng. So sánh đường đi đó với hmac.new(...).digest(), đồng thời kiểm tra _ssl.c quanh dòng 3449 để tìm cách sử dụng libctx mặc định tương tự. Hoàn tất có nghĩa là xác lập được một lỗi có thể tái hiện hoặc một chiến lược xử lý an toàn đã được xác nhận cho các phiên bản OpenSSL bị ảnh hưởng và các triển khai SSL tương thích.