apache / apache/incubator-pegasus
Server crashed in tcmalloc
- Dominant language
- C++
- Stars
- 2.1k
- Forks
- 328
- PR merge metrics
- No merged PRs in 30d
Description
Server crashed with backtrace:
```
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000000000000000 in ?? ()
Installing openjdk unwinder
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so-gdb.py", line 52, in
class Types(object):
File "/usr/share/gdb/auto-load/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so-gdb.py", line 66, in Types
nmethodp_t = gdb.lookup_type('nmethod').pointer()
gdb.error: No type named nmethod.
(gdb)
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00007fd1da732bf0 in (anonymous namespace)::stacktrace_generic_fp::capture (result=result@entry=0x560793e14010, max_depth=31, skip_count=1, initial_frame=initial_frame@entry=0x7ffc7bc14de0, initial_pc=initial_pc@entry=0x0, sizes=0x0)
at src/stacktrace_generic_fp-inl.h:175
#2 0x00007fd1da732c4a in GetStackTrace_generic_fp (result=0x560793e14010, max_depth=, skip_count=) at src/stacktrace_generic_fp-inl.h:332
#3 0x00007fd1da732fd2 in GetStackTrace (result=result@entry=0x560793e14010, max_depth=max_depth@entry=30, skip_count=skip_count@entry=0) at src/stacktrace.cc:346
#4 0x00007fd1da72390d in tcmalloc::PageHeap::HandleUnlock (this=0x7fd1da74f700 , context=context@entry=0x7ffc7bc14e70) at src/page_heap.cc:155
#5 0x00007fd1da7253dc in tcmalloc::PageHeap::LockingContext::~LockingContext (this=0x7ffc7bc14e70, __in_chrg=) at src/page_heap.cc:77
#6 tcmalloc::PageHeap::NewWithSizeClass (this=this@entry=0x7fd1da74f700 , n=n@entry=9, sizeclass=62) at src/page_heap.cc:172
#7 0x00007fd1da723010 in tcmalloc::CentralFreeList::Populate (this=this@entry=0x7fd1da8e4f00 ) at src/static_vars.h:76
#8 0x00007fd1da7231e8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (end=0x7ffc7bc14fc0, start=0x7ffc7bc14fb8, N=1, this=0x7fd1da8e4f00 ) at src/central_freelist.cc:273
#9 tcmalloc::CentralFreeList::FetchFromOneSpansSafe (this=0x7fd1da8e4f00 , N=1, start=0x7ffc7bc14fb8, end=0x7ffc7bc14fc0) at src/central_freelist.cc:270
#10 0x00007fd1da723293 in tcmalloc::CentralFreeList::RemoveRange (this=0x7fd1da8e4f00 , start=start@entry=0x7ffc7bc14fb8, end=end@entry=0x7ffc7bc14fc0, N=1) at src/central_freelist.cc:253
#11 0x00007fd1da7271a7 in tcmalloc::ThreadCache::FetchFromCentralCache (this=this@entry=0x560793e34000, cl=cl@entry=62, byte_size=byte_size@entry=73728, oom_handler=oom_handler@entry=0x7fd1da713300 <(anonymous namespace)::nop_oom_handler(size_t)>)
at src/thread_cache.cc:124
#12 0x00007fd1da73671d in tcmalloc::ThreadCache::Allocate (oom_handler=0x7fd1da713300 <(anonymous namespace)::nop_oom_handler(size_t)>, cl=62, size=73728, this=0x560793e34000) at src/thread_cache.h:381
#13 (anonymous namespace)::do_malloc (size=72704) at src/tcmalloc.cc:1414
#14 tcmalloc::do_allocate_full (size=72704) at src/tcmalloc.cc:1804
#15 tcmalloc::allocate_full_malloc_oom (size=72704) at src/tcmalloc.cc:1820
#16 0x00007fd1d9e9997a in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#17 0x00007fd1df54947e in ?? () from /lib64/ld-linux-x86-64.so.2
#18 0x00007fd1df549568 in ?? () from /lib64/ld-linux-x86-64.so.2
#19 0x00007fd1df5632ea in ?? () from /lib64/ld-linux-x86-64.so.2
#20 0x0000000000000004 in ?? ()
#21 0x00007ffc7bc1583e in ?? ()
#22 0x00007ffc7bc15878 in ?? ()
#23 0x00007ffc7bc15883 in ?? ()
#24 0x00007ffc7bc1588d in ?? ()
#25 0x0000000000000000 in ?? ()
(gdb)
```
Not sure why it happen, but I guess there are some ralationship of bumping the gperftools to 2.13 (https://github.com/apache/incubator-pegasus/commit/3238686bf4ee062f819521817fc43f39bd70a548).
Contributor guide
Research direction
Start by comparing the gperftools 2.13 bump in commit 3238686bf4ee062f819521817fc43f39bd70a548 with the failing path in src/stacktrace_generic_fp-inl.h, src/page_heap.cc, and src/central_freelist.cc. Use the supplied backtrace to investigate the SIGSEGV during allocation; done means the cause is identified and the crash is prevented or reproducibly resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100