another envoy shut down issue: assert numSymbols() == 0
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 437
Description
The signal-unsafe can be ignored for now. I paste for keep the context self-contained.
The broken assertion is probably '#18'.
My guess is that this is wasm specific issue.
```
#18 Envoy::Stats::SymbolTableImpl::~SymbolTableImpl() (envoy+0xd128b9f)
#19 Envoy::MainCommonBase::~MainCommonBase() (envoy+0x5a5e8c7)
#20 Envoy::MainCommon::~MainCommon() (envoy+0x5a5e789)
#21 std::__1::default_delete::operator()(Envoy::MainCommon*) const (envoy+0x5a5e716)
#22 std::__1::unique_ptr >::reset(Envoy::MainCommon*) (envoy+0x5a5e640
```
``` Full sequence
2021/02/24 21:16:04 stop envoy ...
2021/02/24 21:16:04 HTTP POST http://127.0.0.1:20181/quitquitquit
[2021-02-24 21:16:04.216][29505][info][main] [external/envoy/source/server/server.cc:821] shutting down server instance
[2021-02-24 21:16:04.216][29505][info][main] [external/envoy/source/server/server.cc:873] Notifying 0 callback(s) with completion.
[2021-02-24 21:16:04.218][29505][info][main] [external/envoy/source/server/server.cc:767] main dispatch loop exited
[2021-02-24 21:16:04.243][29505][warning][config] [bazel-out/k8-fastbuild/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamAggregatedResources gRPC config stream closed: 13,
[2021-02-24 21:16:04.245][29505][info][main] [external/envoy/source/server/server.cc:813] exiting
2021/02/24 21:16:04 got metrics 2, want 4
2021/02/24 21:16:04 sleeping till next check
[2021-02-24 21:16:04.327][29505][critical][assert] [external/envoy/source/common/stats/symbol_table_impl.cc:194] assert failure: numSymbols() == 0.
[2021-02-24 21:16:04.327][29505][critical][backtrace] [bazel-out/k8-fastbuild/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x7341
[2021-02-24 21:16:04.327][29505][critical][backtrace] [bazel-out/k8-fastbuild/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
2021/02/24 21:16:05 got metrics 2, want 4
2021/02/24 21:16:05 sleeping till next check
==================
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=29505)
#0 malloc (envoy+0x5356884)
#1 operator new(unsigned long) (libc++.so.1+0x86084)
#2 std::__1::allocator::allocate(unsigned long, void const*) (envoy+0x53cb6b4)
#3 std::__1::allocator_traits >::allocate(std::__1::allocator&, unsigned long) (envoy+0x53cb42a)
#4 fmt::v7::basic_memory_buffer >::grow(unsigned long) (envoy+0x53caf41)
#5 fmt::v7::detail::buffer::reserve(unsigned long) (envoy+0x53cefc0)
#6 void fmt::v7::detail::buffer::append(char const*, char const*) (envoy+0x53ced52)
#7 spdlog::details::fmt_helper::append_string_view(fmt::v7::basic_string_view, fmt::v7::basic_memory_buffer >&) (envoy+0x53ce79e)
#8 spdlog::details::v_formatter::format(spdlog::details::log_msg const&, tm const&, fmt::v7::basic_memory_buffer >&) (envoy+0x543d3bf)
#9 spdlog::pattern_formatter::format(spdlog::details::log_msg const&, fmt::v7::basic_memory_buffer >&) (envoy+0x53c9947)
#10 Envoy::Logger::DelegatingLogSink::log(spdlog::details::log_msg const&) (envoy+0xec120dc)
#11 spdlog::logger::sink_it_(spdlog::details::log_msg const&) (envoy+0x53c91db)
#12 spdlog::logger::log_it_(spdlog::details::log_msg const&, bool, bool) (envoy+0x589141a)
#13 void spdlog::logger::log_, std::__1::basic_string, std::__1::allocator > >(spdlog::source_loc, spdlog::level::level_enum, fmt::v7::basic_string_view const&, std::__1::basic_string, std::__1::allocator > const&) (envoy+0x58bd7e9)
#14 void spdlog::logger::log, std::__1::allocator > >(spdlog::source_loc, spdlog::level::level_enum, fmt::v7::basic_string_view, std::__1::basic_string, std::__1::allocator > const&) (envoy+0x58ba891)
#15 Envoy::BackwardsTrace::logTrace() (envoy+0xcc20d78)
#16 Envoy::SignalAction::sigHandler(int, siginfo_t*, void*) (envoy+0xcc1d532)
#17 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) (envoy+0x535e9ef)
#18 Envoy::Stats::SymbolTableImpl::~SymbolTableImpl() (envoy+0xd128b9f)
#19 Envoy::MainCommonBase::~MainCommonBase() (envoy+0x5a5e8c7)
#20 Envoy::MainCommon::~MainCommon() (envoy+0x5a5e789)
#21 std::__1::default_delete::operator()(Envoy::MainCommon*) const (envoy+0x5a5e716)
#22 std::__1::unique_ptr >::reset(Envoy::MainCommon*) (envoy+0x5a5e640)
#23 std::__1::unique_ptr >::~unique_ptr() (envoy+0x5a3783c)
#24 Envoy::MainCommon::main(int, char**, std::__1::function) (envoy+0x5a312ee)
#25 main (envoy+0x5a2d6f5)
```
Contributor guide
Assessment
This issue has not been assessed yet.