rr-debugger / rr-debugger/rr

rr record failed with SEGV

Open
#3,292 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
10.7k
Forks
662
Avg merge
2d 3h
Merged PRs (30d)
2

Description

In rr here was what I saw:

Before executing 0x00007fc0c40def2d, the assembly looked like this:
0x00007fc0c40deeeb <+427>: syscall
0x00007fc0c40deeed <+429>: jmp 0x7fc0c40def2f <__pthread_cond_broadcast+495>
0x00007fc0c40deeef <+431>: mov (%r9),%eax
0x00007fc0c40deef2 <+434>: data16 data16 data16 data16 nopw %cs:0x0(%rax,%rax,1)
0x00007fc0c40def00 <+448>: mov %eax,%ecx
0x00007fc0c40def02 <+450>: and $0xfffffffc,%ecx
0x00007fc0c40def05 <+453>: lock cmpxchg %ecx,(%r9)
0x00007fc0c40def0a <+458>: jne 0x7fc0c40def00 <__pthread_cond_broadcast+448>
0x00007fc0c40def0c <+460>: and $0x3,%eax
0x00007fc0c40def0f <+463>: cmp $0x2,%eax
0x00007fc0c40def12 <+466>: mov -0x2c(%rbp),%esi
0x00007fc0c40def15 <+469>: jne 0x7fc0c40def3b <__pthread_cond_broadcast+507>
0x00007fc0c40def17 <+471>: xor $0x81,%esi
0x00007fc0c40def1d <+477>: mov $0xca,%eax
0x00007fc0c40def22 <+482>: mov %r9,%rdi
0x00007fc0c40def25 <+485>: mov $0x1,%edx
0x00007fc0c40def2a <+490>: xor %r10d,%r10d
=> 0x00007fc0c40def2d <+493>: syscall
0x00007fc0c40def2f <+495>: cmp $0xfffffffffffff001,%rax
0x00007fc0c40def35 <+501>: jae 0x7fc0c40df012 <__pthread_cond_broadcast+722>
0x00007fc0c40def3b <+507>: xor %eax,%eax
0x00007fc0c40def3d <+509>: add $0x8,%rsp

The syscall at 0x7fc0c40def2d was a futex_wake and was generated from the source code here:
https://github.com/bminor/glibc/blob/b92a49359f33a461db080a33940d73f47c756126/nptl/pthread_cond_broadcast.c#L85.

After executing 0x00007fc0c40def2d, the mem locations from 0x00007fc0c40def2d to 0x00007fc0c40def34 were rewritten.

0x00007fc0c40deeeb <+427>: syscall
0x00007fc0c40deeed <+429>: jmp 0x7fc0c40def2f <__pthread_cond_broadcast+495>
0x00007fc0c40deeef <+431>: mov (%r9),%eax
0x00007fc0c40deef2 <+434>: data16 data16 data16 data16 nopw %cs:0x0(%rax,%rax,1)
0x00007fc0c40def00 <+448>: mov %eax,%ecx
0x00007fc0c40def02 <+450>: and $0xfffffffc,%ecx
0x00007fc0c40def05 <+453>: lock cmpxchg %ecx,(%r9)
0x00007fc0c40def0a <+458>: jne 0x7fc0c40def00 <__pthread_cond_broadcast+448>
0x00007fc0c40def0c <+460>: and $0x3,%eax
0x00007fc0c40def0f <+463>: cmp $0x2,%eax
0x00007fc0c40def12 <+466>: mov -0x2c(%rbp),%esi
0x00007fc0c40def15 <+469>: jne 0x7fc0c40def3b <__pthread_cond_broadcast+507>
0x00007fc0c40def17 <+471>: xor $0x81,%esi
0x00007fc0c40def1d <+477>: mov $0xca,%eax
0x00007fc0c40def22 <+482>: mov %r9,%rdi
0x00007fc0c40def25 <+485>: mov $0x1,%edx
0x00007fc0c40def2a <+490>: xor %r10d,%r10d
0x00007fc0c40def2d <+493>: jmpq 0x7fc0c40cdcbe
0x00007fc0c40def32 <+498>: nop
0x00007fc0c40def33 <+499>: nop
0x00007fc0c40def34 <+500>: nop
=> 0x00007fc0c40def35 <+501>: jae 0x7fc0c40df012 <__pthread_cond_broadcast+722>
0x00007fc0c40def3b <+507>: xor %eax,%eax
0x00007fc0c40def3d <+509>: add $0x8,%rsp

Then the program executed 0x00007fc0c40deeed a little bit later where the jmp target was 0x7fc0c40def2f. Now 0x7fc0c40def2f was not an instruction any more. I would assume I would get a SIGILL but actually I get a SEGV here.

0x00007fc0c40deeeb <+427>: syscall
=> 0x00007fc0c40deeed <+429>: jmp 0x7fc0c40def2f <__pthread_cond_broadcast+495>
0x00007fc0c40deeef <+431>: mov (%r9),%eax
0x00007fc0c40deef2 <+434>: data16 data16 data16 data16 nopw %cs:0x0(%rax,%rax,1)
0x00007fc0c40def00 <+448>: mov %eax,%ecx
0x00007fc0c40def02 <+450>: and $0xfffffffc,%ecx
0x00007fc0c40def05 <+453>: lock cmpxchg %ecx,(%r9)
0x00007fc0c40def0a <+458>: jne 0x7fc0c40def00 <__pthread_cond_broadcast+448>
0x00007fc0c40def0c <+460>: and $0x3,%eax
0x00007fc0c40def0f <+463>: cmp $0x2,%eax
0x00007fc0c40def12 <+466>: mov -0x2c(%rbp),%esi
0x00007fc0c40def15 <+469>: jne 0x7fc0c40def3b <__pthread_cond_broadcast+507>
0x00007fc0c40def17 <+471>: xor $0x81,%esi
0x00007fc0c40def1d <+477>: mov $0xca,%eax
0x00007fc0c40def22 <+482>: mov %r9,%rdi
0x00007fc0c40def25 <+485>: mov $0x1,%edx
0x00007fc0c40def2a <+490>: xor %r10d,%r10d
0x00007fc0c40def2d <+493>: jmpq 0x7fc0c40cdcbe
0x00007fc0c40def32 <+498>: nop
0x00007fc0c40def33 <+499>: nop
0x00007fc0c40def34 <+500>: nop
0x00007fc0c40def35 <+501>: jae 0x7fc0c40df012 <__pthread_cond_broadcast+722>
0x00007fc0c40def3b <+507>: xor %eax,%eax
0x00007fc0c40def3d <+509>: add $0x8,%rsp

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the rr recording failure and compare the syscall bytes before and after execution, using the glibc pthread_cond_broadcast.c reference linked in the report as context. The issue does not name an rr source file or test; done would require identifying why the recorded instruction bytes are rewritten and adding a regression test or documented fix for the resulting SEGV.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
devtools, operating-systems, reverse-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.