envoyproxy / envoyproxy/envoy

Crash in Envoy::Http::AsyncStreamImpl::closeRemote on shutdown

Ouverte
#7,983 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
bug help wanted
Langage dominant
C++
Étoiles
28.9k
Forks
5.6k
Merge moyen
1 j 22 h
PR mergées (30 j)
430

Description

Seeing the following crash running off 38a4d86407668612ccbc4cba65d11be6cbd81b32:

```
2019-08-20_22:24:17.36421 [2019-08-20 22:24:17.364][374905][warning][main] [external/envoy/source/server/server.cc:469] caught SIGTERM
2019-08-20_22:24:17.36423 [2019-08-20 22:24:17.364][374905][info][main] [external/envoy/source/server/server.cc:574] shutting down server instance
2019-08-20_22:24:17.36424 [2019-08-20 22:24:17.364][374905][info][main] [external/envoy/source/server/server.cc:528] main dispatch loop exited
2019-08-20_22:24:17.36959 [2019-08-20 22:24:17.369][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:81] Caught Segmentation f
ault, suspect faulting address 0x0
2019-08-20_22:24:17.36962 [2019-08-20 22:24:17.369][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:69] Backtrace (use tools/
stack_decode.py to get line numbers):
2019-08-20_22:24:17.37594 [2019-08-20 22:24:17.375][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #0: __restore_rt [0x7
f1f290895d0]
2019-08-20_22:24:17.39615 [2019-08-20 22:24:17.396][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #1: Envoy::Http::AsyncStreamImpl::closeRemote() [0x8f0b46]
2019-08-20_22:24:17.40303 [2019-08-20 22:24:17.402][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #2: Envoy::Http::AsyncStreamImpl::encodeData() [0x8f5148]
2019-08-20_22:24:17.41097 [2019-08-20 22:24:17.410][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #3: Envoy::Http::Utility::sendLocalReply() [0xa05ff8]
2019-08-20_22:24:17.41454 [2019-08-20 22:24:17.414][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #4: Envoy::Http::AsyncStreamImpl::sendLocalReply() [0x8f0e69]
2019-08-20_22:24:17.41906 [2019-08-20 22:24:17.419][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #5: Envoy::Router::Filter::onUpstreamAbort() [0x8f71c3]
2019-08-20_22:24:17.42254 [2019-08-20 22:24:17.422][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #6: Envoy::Router::Filter::onUpstreamReset() [0x902bee]
2019-08-20_22:24:17.42603 [2019-08-20 22:24:17.425][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #7: Envoy::Router::Filter::UpstreamRequest::onResetStream() [0x902fb2]
2019-08-20_22:24:17.42950 [2019-08-20 22:24:17.429][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #8: Envoy::Http::Http1::ClientConnectionImpl::onResetStream() [0x95f30d]
2019-08-20_22:24:17.43457 [2019-08-20 22:24:17.434][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #9: Envoy::Http::CodecClient::onEvent() [0x8e27a4]
2019-08-20_22:24:17.44230 [2019-08-20 22:24:17.442][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #10: Envoy::Network::ConnectionImpl::raiseEvent() [0x753b25]
2019-08-20_22:24:17.44620 [2019-08-20 22:24:17.446][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #11: Envoy::Network::ConnectionImpl::closeSocket() [0x753045]
2019-08-20_22:24:17.45422 [2019-08-20 22:24:17.454][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #12: Envoy::Network::ConnectionImpl::close() [0x75489d]
2019-08-20_22:24:17.45843 [2019-08-20 22:24:17.458][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #13: Envoy::Http::Http1::ConnPoolImpl::~ConnPoolImpl() [0x88abf5]
2019-08-20_22:24:17.46249 [2019-08-20 22:24:17.462][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #14: Envoy::Http::Http1::ProdConnPoolImpl::~ProdConnPoolImpl() [0x88ad00]
2019-08-20_22:24:17.46651 [2019-08-20 22:24:17.466][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #15: Envoy::Upstream::ConnPoolMap<>::clearActivePools() [0x78500e]
2019-08-20_22:24:17.46989 [2019-08-20 22:24:17.469][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #16: Envoy::Upstream::ConnPoolMap<>::~ConnPoolMap() [0x78c2e5]
2019-08-20_22:24:17.47316 [2019-08-20 22:24:17.473][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #17: std::_Sp_counted_ptr_inplace<>::_M_dispose() [0x78c428]
2019-08-20_22:24:17.47640 [2019-08-20 22:24:17.476][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #18: std::_Hashtable<>::clear() [0x784e4a]
2019-08-20_22:24:17.47980 [2019-08-20 22:24:17.479][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #19: Envoy::Upstream::ClusterManagerImpl::ThreadLocalClusterManagerImpl::~ThreadLocalClusterManagerImpl() [0x789c2b]
2019-08-20_22:24:17.48601 [2019-08-20 22:24:17.485][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #20: Envoy::ThreadLoc
al::InstanceImpl::shutdownThread() [0x70873a]
2019-08-20_22:24:17.48990 [2019-08-20 22:24:17.489][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #21: Envoy::Server::W
orkerImpl::threadRoutine() [0x746d49]
2019-08-20_22:24:17.49403 [2019-08-20 22:24:17.493][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #22: Envoy::Thread::T
hreadImplPosix::ThreadImplPosix()::{lambda()#1}::_FUN() [0xc879c5]
2019-08-20_22:24:17.49410 [2019-08-20 22:24:17.494][376231][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #23: start_thread [0x
7f1f29081dd5]
2019-08-20_22:24:17.49412 AsyncClient 0x4981808, stream_id_: 1003858367269869698
2019-08-20_22:24:17.49413 &stream_info_:
2019-08-20_22:24:17.49413 StreamInfoImpl 0x4981960, protocol_: 1, response_code_: null, response_code_details_: upstream_reset_before_response_started{connection termination}, health_check_request_: 0, rout
e_name_:
```

Not 100% sure that this was introduced with our recent update, as we've been seeing crashes with similar traces on older versions.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.