SIGSEGV in memtx.perftest

Bug description

• OS: Linux
• OS Version: Ubuntu 22.04
• Architecture: amd64

Tarantool 3.2.0-entrypoint-161-g030e64803e
Target: Linux-x86_64-RelWithDebInfo
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=TRUE
Compiler: GNU-11.4.0
C_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -O2 -g -DNDEBUG -ggdb -O2
CXX_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c++11 -Wall -Wextra -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -O2 -g -DNDEBUG -ggdb -O2

Steps to reproduce

$ ./build/perf/memtx.perftest 
2024-07-20T20:36:52+03:00
Running ./build/perf/memtx.perftest
Run on (8 X 1185.99 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 12288 KiB (x1)
Load Average: 0.88, 0.68, 0.45
mapping 1073741824 bytes for memtx tuple arena...

<snipped>
$ cd perf/lua/
$ ../../build/perf/memtx.perftest 
2024-07-20T20:37:49+03:00
Running ../../build/perf/memtx.perftest
Run on (8 X 854.655 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 12288 KiB (x1)
Load Average: 1.03, 0.77, 0.50
Segmentation fault (core dumped)
$ rm 00000000000000001003.snap 
$ ../../build/perf/memtx.perftest 
2024-07-20T20:38:50+03:00
Running ../../build/perf/memtx.perftest
Run on (8 X 1644.97 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 12288 KiB (x1)
Load Average: 1.09, 0.84, 0.54
mapping 1073741824 bytes for memtx tuple arena...
Actual slab_alloc_factor calculated on the basis of desired slab_alloc_factor = 1.090508
fiber has not yielded for more than 0.500 seconds
-----------------------------------------------------------------------------------------------------
Benchmark                                           Time             CPU   Iterations UserCounters...
-----------------------------------------------------------------------------------------------------
MemtxFixture/TreeGetRandomExistingKeys            802 ns          802 ns       909464 items_per_second=1.24701M/s
MemtxFixture/TreeGet1RandomExistingKey            404 ns          404 ns      1641841 items_per_second=2.4759M/s
MemtxFixture/TreeSelectAll                    6439206 ns      6436792 ns          107 items_per_second=20.3629M/s
MemtxFixture/TreeReplaceRandomExistingKeys       1260 ns         1257 ns       576237 items_per_second=795.323k/s

Actual behavior

I see a segmentation fault in memtx.perftest when it is executed in a directory with existed tarantool files.

(gdb) run
Starting program: /home/sergeyb/sources/MRG/tarantool/build/perf/memtx.perftest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
2024-07-20T17:43:35+03:00
Running /home/sergeyb/sources/MRG/tarantool/build/perf/memtx.perftest
Run on (8 X 721.996 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 12288 KiB (x1)
Load Average: 0.39, 0.86, 0.97

Program received signal SIGSEGV, Segmentation fault.
rlist_add_tail (item=0x555555c56760, head=0x555555ba2450 <gc+304>) at /home/sergeyb/sources/MRG/tarantool/src/lib/small/include/small/rlist.h:87
87              item->prev->next = item;
(gdb) 
(gdb) bt
#0  rlist_add_tail (item=0x555555c56760, head=0x555555ba2450 <gc+304>)
    at /home/sergeyb/sources/MRG/tarantool/src/lib/small/include/small/rlist.h:87
#1  gc_add_checkpoint (vclock=0x555555c75e90) at /home/sergeyb/sources/MRG/tarantool/src/box/gc.c:485
#2  0x000055555560eeb8 in memtx_engine_new (snap_dirname=<optimized out>, force_recovery=<optimized out>, tuple_arena_max_size=1073741824, 
    objsize_min=16, dontdump=<optimized out>, granularity=8, allocator=0x555555ad6328 "small", alloc_factor=1.10000002, sort_threads=0, 
    on_indexes_built=0x5555555e4b10 <Memtx::memtx_on_indexes_built_mock_cb()>) at /home/sergeyb/sources/MRG/tarantool/src/box/memtx_engine.cc:1694
#3  0x00005555555e51b4 in Memtx::Memtx (this=0x555555b99c20 <Memtx::instance()::singleton>)
    at /home/sergeyb/sources/MRG/tarantool/perf/memtx.cc:124
#4  0x00005555555e60df in Memtx::instance () at /home/sergeyb/sources/MRG/tarantool/perf/memtx.cc:85
#5  MemtxFixture::SetUp (this=0x555555c3d550) at /home/sergeyb/sources/MRG/tarantool/perf/memtx.cc:221
#6  0x00005555555e4c54 in benchmark::Fixture::SetUp (st=..., this=0x555555c3d550) at /usr/include/benchmark/benchmark.h:1225
#7  benchmark::Fixture::Run (this=0x555555c3d550, st=...) at /usr/include/benchmark/benchmark.h:1216
#8  0x00007ffff7f60a28 in benchmark::internal::BenchmarkInstance::Run(unsigned long, int, benchmark::internal::ThreadTimer*, benchmark::internal::ThreadManager*, benchmark::internal::PerfCountersMeasurement*) const () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#9  0x00007ffff7f7a8e1 in ?? () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#10 0x00007ffff7f7b8a5 in benchmark::internal::BenchmarkRunner::DoNIterations() () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#11 0x00007ffff7f7fa6c in benchmark::internal::BenchmarkRunner::DoOneRepetition() () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#12 0x00007ffff7f6949a in ?? () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#13 0x00007ffff7f6acee in benchmark::RunSpecifiedBenchmarks(benchmark::BenchmarkReporter*, benchmark::BenchmarkReporter*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#14 0x00007ffff7f6aecc in benchmark::RunSpecifiedBenchmarks() () from /lib/x86_64-linux-gnu/libbenchmark.so.1
#15 0x00005555555e3408 in main (argc=<optimized out>, argv=0x7fffffffdd28) at /home/sergeyb/sources/MRG/tarantool/perf/memtx.cc:390
(gdb) 

Expected behavior

no sigsegv

Reproduce the crash by running build/perf/memtx.perftest from a directory containing existing Tarantool files, then inspect perf/memtx.cc and the stack through src/box/memtx_engine.cc, src/box/gc.c, and src/lib/small/include/small/rlist.h. Done means the benchmark no longer raises SIGSEGV in that situation and still runs when the directory is clean.