Data races in OpenSSL bindings

Bug report

We have some data races in our OpenSSL bindings that we were not previously detecting because we didn't compile OpenSSL with TSan (https://github.com/python/cpython/issues/143750):

test_load_cert_chain_thread_safety

Race between use_certificate_chain_file and SSL_CTX_set_default_passwd_cb:

WARNING: ThreadSanitizer: data race (pid=3543367)
  Read of size 8 at 0x726c000007b8 by thread T6:
    #0 use_certificate_chain_file /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_rsa.c:477:32 (libssl.so.3+0x80b0f) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #1 SSL_CTX_use_certificate_chain_file /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_rsa.c:587:12 (libssl.so.3+0x80a19) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #2 _ssl__SSLContext_load_cert_chain_impl /home/sgross/cpython/./Modules/_ssl.c:4595:9 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x1f9b8) (BuildId: b7a1f8db907210c3a84626c1dff3c67d6f138a0a)
    #3 _ssl__SSLContext_load_cert_chain /home/sgross/cpython/./Modules/clinic/_ssl.c.h:1833:20 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x1f9b8)
...

  Previous write of size 8 at 0x726c000007b8 by thread T4:
    #0 SSL_CTX_set_default_passwd_cb /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:4497:34 (libssl.so.3+0x6c654) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #1 _ssl__SSLContext_load_cert_chain_impl /home/sgross/cpython/./Modules/_ssl.c:4640:5 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x1fd64) (BuildId: b7a1f8db907210c3a84626c1dff3c67d6f138a0a)
    #2 _ssl__SSLContext_load_cert_chain /home/sgross/cpython/./Modules/clinic/_ssl.c.h:1833:20 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x1fd64)

test_thread_recv_while_main_thread_sends

WARNING: ThreadSanitizer: data race (pid=3543528)
  Write of size 4 at 0x72880000f068 by main thread:
    #0 ssl3_write_bytes /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/record/rec_layer_s3.c:285:16 (libssl.so.3+0x14050b) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #1 ssl3_write /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/s3_lib.c:4648:12 (libssl.so.3+0x3eee9) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #2 ssl_write_internal /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:2604:16 (libssl.so.3+0x65007) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #3 SSL_write_ex2 /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:2707:15 (libssl.so.3+0x653b5) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #4 SSL_write_ex /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:2701:12 (libssl.so.3+0x65339) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #5 _ssl__SSLSocket_write_impl /home/sgross/cpython/./Modules/_ssl.c:2792:18 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x2a809) (BuildId: b7a1f8db907210c3a84626c1dff3c67d6f138a0a)
    #6 _ssl__SSLSocket_write /home/sgross/cpython/./Modules/clinic/_ssl.c.h:651:20 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x2a809)
    #7 _PyEval_EvalFrameDefault /home/sgross/cpython/Python/generated_cases.c.h:4009:35 (python+0x5129d9) (BuildId: 09bdcea4d998e789f62d66f1eea397f4c0970fe2)
...

  Previous write of size 4 at 0x72880000f068 by thread T492:
    #0 ssl3_read_bytes /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/record/rec_layer_s3.c:681:16 (libssl.so.3+0x141f69) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #1 ssl3_read_internal /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/s3_lib.c:4666:9 (libssl.so.3+0x3f2ca) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #2 ssl3_read /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/s3_lib.c:4689:12 (libssl.so.3+0x3f127) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #3 ssl_read_internal /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:2379:16 (libssl.so.3+0x63af1) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #4 SSL_read_ex /home/sgross/cpython/multissl/src/openssl-3.5.4/ssl/ssl_lib.c:2407:15 (libssl.so.3+0x64225) (BuildId: 3439d4624f0bbda0d0e957787b11e780635293c6)
    #5 _ssl__SSLSocket_read_impl /home/sgross/cpython/./Modules/_ssl.c:2958:18 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x2b229) (BuildId: b7a1f8db907210c3a84626c1dff3c67d6f138a0a)
    #6 _ssl__SSLSocket_read /home/sgross/cpython/./Modules/clinic/_ssl.c.h:723:20 (_ssl.cpython-315td-x86_64-linux-gnu.so+0x2b229)
    #7 method_vectorcall_VARARGS /home/sgross/cpython/Objects/descrobject.c:325:24 (python+0x293161) (BuildId: 09bdcea4d998e789f62d66f1eea397f4c0970fe2)

Linked PRs

• gh-143816
• gh-143818

まず test_load_cert_chain_thread_safety と test_thread_recv_while_main_thread_sends のケースから始め、次に対応する _ssl.c の呼び出し箇所と ThreadSanitizer のトレースを調べます。OpenSSL/CPython のテストを TSan 下で実行します。報告されたデータ競合が現れなくなれば完了です。リンクされた PRs が現在の作業コンテキストを提供します。