tarantool / tarantool/tarantool

Incorrect streaming in sysprof when fast function is on top

Open
#11,664 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug crash luajit sysprof
Dominant language
Lua
Stars
3.7k
Forks
419
Avg merge
1d 23h
Merged PRs (30d)
88

Description

How to reproduce (isn't stable).

Apply the following patch:

diff --git a/test/config-luatest/CMakeLists.txt b/test/config-luatest/CMakeLists.txt
index 6c13cb810..8fac27489 100644
--- a/test/config-luatest/CMakeLists.txt
+++ b/test/config-luatest/CMakeLists.txt
@@ -29,6 +29,7 @@ foreach(test_path ${tests})
                    # Disable stdout buffering,
                    # see https://github.com/tarantool/test-run/issues/119.
                    -e "io.stdout:setvbuf('no')"
+                   -e "misc.sysprof.start({mode = 'C', path = '/dev/null', interval = 1})"
                    ${LUATEST_COMMAND_WITH_FLAGS}
                    --shuffle all:${RANDOM_SEED}
                    ${test_title}

Build Tarantool like the following:

cmake -DCMAKE_BUILD_TYPE=Debug -DLUA_USE_APICHECK=ON -DLUA_USE_ASSERT=ON -DLUAJIT_ENABLE_GC64=OFF -DENABLE_ASAN=OFF . && make tarantool -j

And run

ctest -R test/config-luatest/anonymous_replica_test.lua -V

Backtrace:

(gdb) bt
#0  0x00007f9847ca4f0c in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007f9847c4fb36 in raise () from /lib64/libc.so.6
#2  0x00007f9847c378fa in abort () from /lib64/libc.so.6
#3  0x00005645c2dab9db in crash_signal_cb (signo=11, siginfo=0x5645eccd53f0, context=0x5645eccd52c0)
    at src/lib/core/crash.c:204
#4  <signal handler called>
#5  0x00005645c2e84b48 in stream_lfunc (buf=0x5645c36330d8 <sysprof+24>, func=0x4188f9e0)
    at third_party/luajit/src/lj_sysprof.c:128
#6  0x00005645c2e84cc6 in stream_frame_lua (buf=0x5645c36330d8 <sysprof+24>, frame=0x40bcf068)
    at third_party/luajit/src/lj_sysprof.c:149
#7  0x00005645c2e84eee in stream_backtrace_lua (sp=0x5645c36330c0 <sysprof>, vmstate=2)
    at third_party/luajit/src/lj_sysprof.c:188
#8  0x00005645c2e852fc in stream_guest (sp=0x5645c36330c0 <sysprof>, vmstate=2)
    at third_party/luajit/src/lj_sysprof.c:256
#9  0x00005645c2e85405 in stream_event (sp=0x5645c36330c0 <sysprof>, vmstate=2)
    at third_party/luajit/src/lj_sysprof.c:291
#10 0x00005645c2e8550f in sysprof_record_sample (sp=0x5645c36330c0 <sysprof>, info=0x7f9845680930)
    at third_party/luajit/src/lj_sysprof.c:321
#11 0x00005645c2e855c5 in sysprof_signal_handler (sig=27, info=0x7f9845680930, ctx=0x7f9845680800)
    at third_party/luajit/src/lj_sysprof.c:337
#12 <signal handler called>
#13 0x00005645c2e1bfcb in lj_fff_res () at buildvm_x86.dasc:1792
#14 0x00005645c2e28a2b in lua_pcall (L=0x40bcb4d8, nargs=3, nresults=-1, errfunc=0)
    at third_party/luajit/src/lj_api.c:1173
#15 0x00005645c2d862ed in luaT_call (L=0x40bcb4d8, nargs=3, nreturns=-1) at src/lua/utils.c:708
#16 0x00005645c2d7b08e in lua_fiber_run_f (ap=0x7f9847012228) at src/lua/fiber.c:452
#17 0x00005645c2ac617c in fiber_cxx_invoke(fiber_func, typedef __va_list_tag __va_list_tag *) (f=0x5645c2d7afd1 <lua_fiber_run_f>, ap=0x7f9847012228)
    at src/lib/core/fiber.h:1324
#18 0x00005645c2db7863 in fiber_loop (data=0x0) at src/lib/core/fiber.c:1167
#19 0x00005645c31e9cd3 in coro_init () at third_party/coro/coro.c:108

The top_frame value is the following. Hence, the invalid stack traversing takes place.

(gdb) lj-tv top_frame
fast function #4 ; next

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with third_party/luajit/src/lj_sysprof.c, especially stream_lfunc(), stream_frame_lua(), and stream_backtrace_lua(), then reproduce with the patch to test/config-luatest/CMakeLists.txt and the anonymous_replica_test.lua command. Done means the sysprof run no longer performs invalid stack traversal or crashes when a fast function is on top.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.