rr-debugger / rr-debugger/rr

Assertion `false' failed to hold. Replaying 'SIGNAL: SIGCHLD(async)': expecting tracee signal or trap, but instead at 'futex'

Open
#3,737 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
10.7k
Forks
662
Avg merge
2d 3h
Merged PRs (30d)
2

Description

Triggered while running rr replay using rr commit 2bb38b94.

[FATAL src/ReplaySession.cc:684:check_pending_sig()] 
 (task 209395 (rec:208378) at time 158709)
 -> Assertion `false' failed to hold. Replaying `SIGNAL: SIGCHLD(async)': expecting tracee signal or trap, but instead at `futex' (ticks: 1938776)
Tail of trace dump:
{
  real_time:73223.457379 global_time:158689, event:`SYSCALLBUF_RESET' tid:208828, ticks:453157052
}
{
  real_time:73223.457457 global_time:158690, event:`SYSCALL: poll' (state:EXITING_SYSCALL) tid:208394, ticks:929
rax:0x1 rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x2 rdi:0x7f7bd4e78d00 rbp:0x7f7bd4e78d40 rsp:0x7f7bd38ffdf0 r8:0x0 r9:0x7 r10:0x1 r11:0x246 r12:0x7f7bd4e79640 r13:0x0 r14:0x7f7bd4e78d00 r15:0x7f7bd4e78cff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x7 fs_base:0x7f7bd4e79640 gs_base:0x0
  { tid:208394, addr:0x7f7bd4e78d00, length:0x10 }
}
{
  real_time:73223.457519 global_time:158691, event:`SYSCALLBUF_FLUSH' tid:208394, ticks:1003
  { syscall:'recvmsg', ret:0x630, size:0x6c0, desched:1 }
}
{
  real_time:73223.457524 global_time:158692, event:`SYSCALL: rrcall_notify_control_msg' (state:ENTERING_SYSCALL) tid:208394, ticks:1003
rax:0xffffffffffffffda rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7f7bd4e78550 rbp:0x7f7bd4e78ce0 rsp:0x7f7bd38ffe10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x630 r13:0x0 r14:0x7f7bd4e78550 r15:0x7f7bd4e78cff rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3eb fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.457528 global_time:158693, event:`SYSCALLBUF_RESET' tid:208394, ticks:1003
}
{
  real_time:73223.459051 global_time:158694, event:`SYSCALL: rrcall_notify_control_msg' (state:EXITING_SYSCALL) tid:208394, ticks:1003
rax:0x0 rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7f7bd4e78550 rbp:0x7f7bd4e78ce0 rsp:0x7f7bd38ffe10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x630 r13:0x0 r14:0x7f7bd4e78550 r15:0x7f7bd4e78cff rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3eb fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.459123 global_time:158695, event:`SYSCALL: openat' (state:ENTERING_SYSCALL) tid:208394, ticks:1761
rax:0xffffffffffffffda rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x80041 rsi:0x7f7bd4e773e8 rdi:0xffffff9c rbp:0x7f7bd4e773e8 rsp:0x7f7bd38ffdf0 r8:0x0 r9:0x3010101070100 r10:0x1b6 r11:0x246 r12:0x80041 r13:0x7f7bd4e78aa0 r14:0x7f7bd4e773e8 r15:0x1b6 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x101 fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.460524 global_time:158696, event:`SYSCALL: openat' (state:EXITING_SYSCALL) tid:208394, ticks:1761
rax:0x3c rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x80041 rsi:0x7f7bd4e773e8 rdi:0xffffff9c rbp:0x7f7bd4e773e8 rsp:0x7f7bd38ffdf0 r8:0x0 r9:0x3010101070100 r10:0x1b6 r11:0x246 r12:0x80041 r13:0x7f7bd4e78aa0 r14:0x7f7bd4e773e8 r15:0x1b6 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x101 fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.460598 global_time:158697, event:`SYSCALLBUF_FLUSH' tid:208394, ticks:1845
  { syscall:'getrandom', ret:0x10, size:0x20, desched:1 }
}
{
  real_time:73223.460602 global_time:158698, event:`SYSCALL: kill' (state:ENTERING_SYSCALL) tid:208394, ticks:1845
rax:0xffffffffffffffda rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x13 rdi:0x32fbc rbp:0x7f7bd4e76d70 rsp:0x7f7bd38ffdf0 r8:0x0 r9:0x3010101070100 r10:0x1b6 r11:0x246 r12:0x7f7bd4e77328 r13:0x8000000000000000 r14:0x5f5e100 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.460605 global_time:158699, event:`SYSCALLBUF_RESET' tid:208394, ticks:1845
}
{
  real_time:73223.460641 global_time:158700, event:`SYSCALL: kill' (state:EXITING_SYSCALL) tid:208394, ticks:1845
rax:0x0 rbx:0x7f7bd38fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x13 rdi:0x32fbc rbp:0x7f7bd4e76d70 rsp:0x7f7bd38ffdf0 r8:0x0 r9:0x3010101070100 r10:0x1b6 r11:0x246 r12:0x7f7bd4e77328 r13:0x8000000000000000 r14:0x5f5e100 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.460773 global_time:158701, event:`SYSCALLBUF_FLUSH' tid:208394, ticks:6088
  { syscall:'clock_gettime', ret:0x0, size:0x20 }
  { syscall:'openat', ret:0x57, size:0x10, desched:1 }
  { syscall:'readlinkat', ret:0x11, size:0x21 }
  { syscall:'statx', ret:0x0, size:0x110 }
  { syscall:'lseek', ret:0x0, size:0x10 }
  { syscall:'read', ret:0x162, size:0x172, desched:1 }
  { syscall:'read', ret:0x0, size:0x10, desched:1 }
  { syscall:'close', ret:0x0, size:0x10 }
}
{
  real_time:73223.460783 global_time:158702, event:`SYSCALL: clock_nanosleep' (state:ENTERING_SYSCALL) tid:208394, ticks:6088
rax:0xffffffffffffffda rbx:0x1 rcx:0xffffffffffffffff rdx:0x7f7bd4e76d10 rsi:0x0 rdi:0x0 rbp:0x7f7bd4e77270 rsp:0x7f7bd4e76c70 r8:0x0 r9:0x3 r10:0x7f7bd4e76d10 r11:0x246 r12:0x7f7bd4e76d10 r13:0x0 r14:0x7f7bd4e76d10 r15:0x7fffffffffffffff rip:0x7f7bf90e57f8 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xe6 fs_base:0x7f7bd4e79640 gs_base:0x0
}
{
  real_time:73223.460786 global_time:158703, event:`SYSCALLBUF_RESET' tid:208394, ticks:6088
}
{
  real_time:73223.461028 global_time:158704, event:`SIGNAL: SIGSTOP(async)' tid:208853, ticks:260884711
rax:0x100 rbx:0x3 rcx:0x100 rdx:0x0 rsi:0x88f1375342677800 rdi:0x7f9b4b9a13b8 rbp:0x7f9b4b9a13d0 rsp:0x7f9b4b9a13b0 r8:0x7f9b4b9a1310 r9:0x7f9b483e5a00 r10:0x2 r11:0x537b r12:0x305 r13:0x7f9b48dcc828 r14:0x0 r15:0x7f9b48dcc000 rip:0x7f9b5b53632e eflags:0x206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9b4b9a2640 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x403ade0b6b3a76400012 st7:0x0 ymm0:0xaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ymm1:0x10000ffffffffffffffff00000000 ymm2:0x7f9b417a92c800007f9b417a9270 ymm3:0x7f9b417a8e3000007f9b417a8dd8 ymm4:0x22f360b05302e2c1e1a16120e0a0806 ymm5:0x400305463442403e3434343c3a343432 ymm6:0x34340a083434322a22940a538e66022f ymm7:0x2a22920a538c66022f360b0536342c1e ymm8:0x22f360b05302e2c1e1a16120e0a0806 ymm9:0x0 ymm10:0x3f800000 ymm11:0x0 ymm12:0x0 ymm13:0x42aa0000 ymm14:0x0 ymm15:0x3f800000
}
{
  real_time:73223.461073 global_time:158705, event:`SIGNAL_DELIVERY: SIGSTOP(async)' tid:208853, ticks:260884711
rax:0x100 rbx:0x3 rcx:0x100 rdx:0x0 rsi:0x88f1375342677800 rdi:0x7f9b4b9a13b8 rbp:0x7f9b4b9a13d0 rsp:0x7f9b4b9a13b0 r8:0x7f9b4b9a1310 r9:0x7f9b483e5a00 r10:0x2 r11:0x537b r12:0x305 r13:0x7f9b48dcc828 r14:0x0 r15:0x7f9b48dcc000 rip:0x7f9b5b53632e eflags:0x206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9b4b9a2640 gs_base:0x0
  { tid:208853, addr:0x7f9b4b9a13b0, length:(nil) }
}
{
  real_time:73223.462370 global_time:158706, event:`SYSCALL: futex' (state:EXITING_SYSCALL) tid:208378, ticks:1938776
rax:0xfffffffffffffe00 rbx:0x7f7bd88fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x89 rdi:0x7f7be3f26788 rbp:0x7f7bf6bfead0 rsp:0x7f7bd88ffdf0 r8:0x0 r9:0x7f7bffffffff r10:0x0 r11:0x246 r12:0x7f7bf911e870 r13:0x7f7bf902a250 r14:0x7f7bf6bfea60 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f7bf6bff640 gs_base:0x0
}
{
  real_time:73223.462425 global_time:158707, event:`SIGNAL: SIGCHLD(async)' tid:208378, ticks:1938776
rax:0xfffffffffffffe00 rbx:0x7f7bd88fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x89 rdi:0x7f7be3f26788 rbp:0x7f7bf6bfead0 rsp:0x7f7bd88ffdf0 r8:0x0 r9:0x7f7bffffffff r10:0x0 r11:0x246 r12:0x7f7bf911e870 r13:0x7f7bf902a250 r14:0x7f7bf6bfea60 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f7bf6bff640 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x403ade0b6b3a76400012 st7:0x403ade0b6b3a76400000 ymm0:0x0 ymm1:0x7f7bd01764800000000000000001 ymm2:0x7f7bd3afec9000007f7bd01039c0 ymm3:0x7f7bb8496a600000000000000001 ymm4:0x2f80712f098806577aff56b20fb51ba6 ymm5:0x700007f7bb870b400 ymm6:0x60000000000000000 ymm7:0x7ffcefbc9d408000000000000000 ymm8:0x0 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0x0 ymm14:0x0 ymm15:0x0
}
{
  real_time:73223.462456 global_time:158708, event:`SIGNAL_DELIVERY: SIGCHLD(async)' tid:208378, ticks:1938776
rax:0xca rbx:0x7f7bd88fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x89 rdi:0x7f7be3f26788 rbp:0x7f7bf6bfead0 rsp:0x7f7bd88ffdf0 r8:0x0 r9:0x7f7bffffffff r10:0x0 r11:0x246 r12:0x7f7bf911e870 r13:0x7f7bf902a250 r14:0x7f7bf6bfea60 r15:0x0 rip:0x70000000 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f7bf6bff640 gs_base:0x0
  { tid:208378, addr:0x7f7bd88ffdf0, length:(nil) }
}
{
  real_time:73223.462615 global_time:158709, event:`SIGNAL: SIGCHLD(async)' tid:208378, ticks:1938776
rax:0xfffffffffffffe00 rbx:0x7f7bd88fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x89 rdi:0x7f7be3f26788 rbp:0x7f7bf6bfead0 rsp:0x7f7bd88ffdf0 r8:0x0 r9:0x7f7bffffffff r10:0x0 r11:0x246 r12:0x7f7bf911e870 r13:0x7f7bf902a250 r14:0x7f7bf6bfea60 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f7bf6bff640 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x403ade0b6b3a76400012 st7:0x403ade0b6b3a76400000 ymm0:0x0 ymm1:0x7f7bd01764800000000000000001 ymm2:0x7f7bd3afec9000007f7bd01039c0 ymm3:0x7f7bb8496a600000000000000001 ymm4:0x2f80712f098806577aff56b20fb51ba6 ymm5:0x700007f7bb870b400 ymm6:0x60000000000000000 ymm7:0x7ffcefbc9d408000000000000000 ymm8:0x0 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0x0 ymm14:0x0 ymm15:0x0
}
{
  real_time:73223.462644 global_time:158710, event:`SIGNAL_DELIVERY: SIGCHLD(async)' tid:208378, ticks:1938776
rax:0xca rbx:0x7f7bd88fffa0 rcx:0xffffffffffffffff rdx:0xffffffff rsi:0x89 rdi:0x7f7be3f26788 rbp:0x7f7bf6bfead0 rsp:0x7f7bd88ffdf0 r8:0x0 r9:0x7f7bffffffff r10:0x0 r11:0x246 r12:0x7f7bf911e870 r13:0x7f7bf902a250 r14:0x7f7bf6bfea60 r15:0x0 rip:0x70000000 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x7f7bf6bff640 gs_base:0x0
  { tid:208378, addr:0x7f7bd88ffdf0, length:(nil) }
}
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x5a)[0x555bdd16701a]
rr(_ZN2rr15emergency_debugEPNS_4TaskE+0xe8)[0x555bdd03dd48]
rr(+0xd3c0a)[0x555bdd046c0a]
rr(+0xd3e0f)[0x555bdd046e0f]
rr(+0xd3e49)[0x555bdd046e49]
rr(_ZN2rr13ReplaySession17check_pending_sigEPNS_10ReplayTaskE+0xd6)[0x555bdd0d1d06]
rr(_ZN2rr13ReplaySession16continue_or_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsENS_12TicksRequestENS_13ResumeRequestE+0xa3)[0x555bdd0d8db3]
rr(_ZN2rr13ReplaySession20emulate_async_signalEPNS_10ReplayTaskERKNS0_15StepConstraintsElNS_15remote_code_ptrE+0x59e)[0x555bdd0dc46e]
rr(_ZN2rr13ReplaySession18try_one_trace_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x289)[0x555bdd0e0769]
rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x21d)[0x555bdd0e1fad]
rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandE+0xbf)[0x555bdd0fe9ef]
rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x38b)[0x555bdd02ad7b]
rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0xc9b)[0x555bdd02ca0b]
rr(+0x162a97)[0x555bdd0d5a97]
rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x69a)[0x555bdd0d703a]
rr(main+0x1a1)[0x555bdcfc2c61]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fbcfea29d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fbcfea29e40]
rr(_start+0x25)[0x555bdcfc5635]
=== End rr backtrace

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The failure is reported at src/ReplaySession.cc:684 in ReplaySession::check_pending_sig; start there and reproduce with rr replay using rr commit 2bb38b94, inspecting the supplied trace around global times 158706-158710. Done means replay no longer asserts when SIGCHLD is replayed while the task is at futex, matching the recorded signal or trap expectation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
devtools, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.