Instruction stream corrupted after futex system call
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10.7k
- Forks
- 662
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 2
Description
When recording my application with rr, it crashes consistently with a corrupt stack trace. After carefully stepping through the instructions up to the moment of the crash, it seems as though executing a syscall instruction in funlockfile corrupts the instructions at the site of the system call, crashing the next call to funlockfile.
This is the complete assembly for the function in glibc (2.31):
00000000000661e0 <_IO_funlockfile@@GLIBC_2.2.5>:
661e0: f3 0f 1e fa endbr64
661e4: 48 8b bf 88 00 00 00 mov 0x88(%rdi),%rdi
661eb: 8b 47 04 mov 0x4(%rdi),%eax
661ee: 83 e8 01 sub $0x1,%eax
661f1: 89 47 04 mov %eax,0x4(%rdi)
661f4: 75 35 jne 6622b <_IO_funlockfile@@GLIBC_2.2.5+0x4b>
661f6: 48 c7 47 08 00 00 00 movq $0x0,0x8(%rdi)
661fd: 00
661fe: 64 8b 14 25 18 00 00 mov %fs:0x18,%edx
66205: 00
66206: 85 d2 test %edx,%edx
66208: 75 06 jne 66210 <_IO_funlockfile@@GLIBC_2.2.5+0x30>
6620a: 83 2f 01 subl $0x1,(%rdi)
6620d: c3 retq
6620e: 66 90 xchg %ax,%ax
66210: 87 07 xchg %eax,(%rdi)
66212: 83 f8 01 cmp $0x1,%eax
66215: 7e 14 jle 6622b <_IO_funlockfile@@GLIBC_2.2.5+0x4b>
66217: 45 31 d2 xor %r10d,%r10d
6621a: ba 01 00 00 00 mov $0x1,%edx
6621f: be 81 00 00 00 mov $0x81,%esi
66224: b8 ca 00 00 00 mov $0xca,%eax
66229: 0f 05 syscall
6622b: c3 retq
6622c: 0f 1f 40 00 nopl 0x0(%rax)
This is what I see in gdb (Ubuntu 9.2-0ubuntu1~20.04):
# Instruction pointer is at 0x7fe08258f229, the address of the syscall.
# Examine the next instructions before we execute the syscall.
(rr) x/b 0x7fe08258f229
0x7fe08258f229 <__funlockfile+73>: 0x0f
0x7fe08258f22a <__funlockfile+74>: 0x05
0x7fe08258f22b <__funlockfile+75>: 0xc3
0x7fe08258f22c: 0x0f
0x7fe08258f22d: 0x1f
0x7fe08258f22e: 0x40
0x7fe08258f22f: 0x00
These match the instructions of the assembly in glibc. Now, stepi over the syscall and re-examine the same addresses:
(rr) si
(rr) x/b 0x7fe08258f229
0x7fe08258f229 <__funlockfile+73>: 0xe9
0x7fe08258f22a <__funlockfile+74>: 0xeb
0x7fe08258f22b <__funlockfile+75>: 0xf4
0x7fe08258f22c: 0x18
0x7fe08258f22d: 0x00
0x7fe08258f22e: 0x90
0x7fe08258f22f: 0x90
What's happened!? The instructions at those addresses have been changed. The next call into this function will crash. The new instructions look like a jump to an address that holds a move instruction:
0x7fe08258f224 <__funlockfile+68> mov $0xca,%eax
0x7fe08258f229 <__funlockfile+73> jmpq 0x7fe08271e719
0x7fe08258f22e nop │
0x7fe08258f22f nop
(rr) x/i 0x7fe08271e719
0x7fe08271e719: mov %rsp,0x70001010
Any help or advice here would be greatly appreciated.
Versions in play here:
gcc : Ubuntu 9.3.0-17ubuntu1 20.04
gdb : Ubuntu 9.2-0ubuntu1 20.04
rr : 5.4.0
glibc : Ubuntu GLIBC 2.31-0ubuntu9.2
kernel: 5.8.0-48-generic #54~20.04.1-Ubuntu SMP x86_64
cpu: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the recording around glibc's funlockfile entry point and the futex syscall, then inspect the instruction bytes before and after stepping over the syscall. The issue provides no rr source file or test to target; done means identifying why the syscall site changes and preventing the resulting instruction corruption, with a regression test if the relevant test location can be found.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- devtools, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100