python coredump with libffi 3.4.2

Bug report

Bug description:

I encountered a core dump and the backtrace shows that it relates to ctypes:

#0  0x00007f2b0071bf71 in closure_fcn (cif=0x7f2adcd88de8, resp=0x7ffeeb800a10, args=0x7ffeeb800890, userdata=0x3b) at /usr1/python/python/target/checkout/Python-3.9.11/Modules/_ctypes/callbacks.c:311
#1  0x00007f2b0072ba09 in ffi_closure_unix64_inner () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#2  0x00007f2b0072c1e8 in ffi_closure_unix64 () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#3  0x00007f2add597c13 in LLVMPYObjectCache::notifyObjectCompiled (this=0x3382350, M=0x324a370, MBR=...) at executionengine.cpp:195
#4  0x00007f2adea3e18b in llvm::MCJIT::emitObject(llvm::Module*) [clone .localalias] () from /path/to/python/lib/python3.9/site-packages/llvmlite/binding/libllvmlite.so
#5  0x00007f2adea3ea39 in llvm::MCJIT::generateCodeForModule(llvm::Module*) [clone .localalias] () from /path/to/python/lib/python3.9/site-packages/llvmlite/binding/libllvmlite.so
#6  0x00007f2adea39f30 in llvm::MCJIT::finalizeObject() [clone .localalias] () from /path/to/python/lib/python3.9/site-packages/llvmlite/binding/libllvmlite.so
#7  0x00007f2add5975f5 in LLVMPY_FinalizeObject (EE=0x3335f80) at executionengine.cpp:63
#8  0x00007f2b0072c052 in ffi_call_unix64 () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#9  0x00007f2b0072b428 in ffi_call_int () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#10 0x00007f2b0072b491 in ffi_call () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so

I changed nothing but the version of libffi from 3.3 to 3.4.2, while it runs ok with version 3,3.
I compile libffi and recompiling python with it:

# compile libffi
./configure --prefix=/path/to/libffi/ CFLAGS="-fPIC" --disable-shared --disable-docs
make && make install

# compile python with libffi
sed -i '/Linux\*|GNU\*) LINKFORSHARED=/ s/="/&-pie /' configure
./configure CFLAGS="-fstack-protector-strong -fPIC -D_FORTIFY_SOURCE=2 -O2" LDFLAGS="-L/path/to/libffi/lib64 -Bstatic -Wl,--build-id=none,-z,noexecstack,-z,relro,-z,now" LIBS="-lffi"
make && make install

I also run unittest of ctypes as ctypes unittest crashes with libffi 3.4.2,

test_callbacks (ctypes.test.test_as_parameter.AsParamPropertyWrapperTestCase) ... Aborted (core dumped)

it cored as follows:

#0  0x00007fa96efcc77b in raise () from /usr/lib64/libc.so.6
#1  0x00007fa96efcdaa1 in abort () from /usr/lib64/libc.so.6
#2  0x00007fa961fa4d5a in dlfree () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#3  0x00007fa961fa5b0e in ffi_closure_free () from /path/to/python/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#4  0x00007fa961f97f24 in CThunkObject_dealloc (myself=0x7fa9616f19e0) at /usr1/python/python/target/checkout/Python-3.9.11/Modules/_ctypes/callbacks.c:23
#5  0x00000000004cb315 in _Py_DECREF (op=<optimized out>) at ./Include/object.h:430
#6  _Py_XDECREF (op=<optimized out>) at ./Include/object.h:497
#7  free_keys_object (keys=0x7fa9616f1870) at Objects/dictobject.c:598
#8  0x00000000004cbbc0 in dictkeys_decref (dk=0x7fa9616f1870) at Objects/dictobject.c:333
#9  dict_dealloc (mp=0x7fa961bdc680) at Objects/dictobject.c:2026

Do you have any ideas?

Tested with:

• python 3.9.11
• libffi 3.4.2
• OS: x86_64 GNU/Linux

CPython versions tested on:

3.9

Operating systems tested on:

Linux

まず Python 3.9.11 と libffi 3.4.2 でクラッシュを再現し、次に libffi 3.3 と比較します。Modules/_ctypes/callbacks.c の closure_fcn と CThunkObject_dealloc 周辺を調査し、ctypes の test_callbacks という名前のテストを実行します。特に test_as_parameter.AsParamPropertyWrapperTestCase での失敗を確認してください。クラッシュの原因が説明され、関連する ctypes テストがパスすれば完了です。