open-telemetry / open-telemetry/opentelemetry-cpp

SetTracerProvider with a new NoopProvider wrapped in a type-inferred nostd::shared_ptr causes segfaults in Release builds

Open
#1,470 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug do-not-stale help wanted
Dominant language
C++
Stars
1.4k
Forks
632
Avg merge
1d 13h
Merged PRs (30d)
75

Description

Environment

GCC 9.4.0
Compiler flags: -O3 -DNDEBUG (CMake Release) or -O2 -g -DNDEBUG (CMake RelWithDebInfo)
Default CMake options, linking only to opentelemetry_trace
OpenTelemetry C++ 1.4.1

Steps to reproduce
#include <opentelemetry/trace/provider.h>

int main(int argc, char** argv) {
  opentelemetry::trace::Provider::SetTracerProvider(
      opentelemetry::nostd::shared_ptr(new opentelemetry::trace::NoopTracerProvider));
  auto t = opentelemetry::trace::Provider::GetTracerProvider()->GetTracer("test");
  t->StartSpan("test");
}
Expected behavior

No segfault.

Actual behavior

Results in a segfault with stacktrace:

std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release shared_ptr_base.h:148
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release shared_ptr_base.h:148
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count shared_ptr_base.h:730
std::__shared_ptr<opentelemetry::v1::trace::TracerProvider, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr shared_ptr_base.h:1169
std::shared_ptr<opentelemetry::v1::trace::TracerProvider>::~shared_ptr shared_ptr.h:103
opentelemetry::v1::nostd::shared_ptr<opentelemetry::v1::trace::TracerProvider>::shared_ptr_wrapper::~shared_ptr_wrapper shared_ptr.h:43
opentelemetry::v1::nostd::shared_ptr<opentelemetry::v1::trace::TracerProvider>::operator= shared_ptr.h:115
opentelemetry::v1::trace::Provider::SetTracerProvider provider.h:40
main noop_provider.cpp:4
__libc_start_call_main 0x00007ffff7b4fd90
__libc_start_main_impl 0x00007ffff7b4fe40
_start 0x00005555555567a5
Additional context

To fix this I eventually figured out explicitly specifying the nostd::shared_ptr template argument fixes the problem:

#include <opentelemetry/trace/provider.h>

int main(int argc, char** argv) {
  opentelemetry::trace::Provider::SetTracerProvider(
      opentelemetry::nostd::shared_ptr<opentelemetry::trace::TracerProvider>(
          new opentelemetry::trace::NoopTracerProvider));
  auto t = opentelemetry::trace::Provider::GetTracerProvider()->GetTracer("test");
  t->StartSpan("test");
}

In a larger system where we've been using the default SDK TracerProvider and after we try to reset it like this, there is actually a deadlock in SetTracerProvider instead of a segfault, for which I haven't been able to produce a minimal example yet, but these deadlocks also disappear with the above fix.

This bug does not occur when building only with -g (CMake Debug).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with opentelemetry/trace/provider.h and opentelemetry/nostd/shared_ptr.h, then reproduce the sample with GCC 9.4.0 using CMake Release or RelWithDebInfo flags. Compare the type-inferred and explicitly typed shared_ptr cases, including the reported stack trace; done means the Release build avoids the segfault and the larger-system deadlock scenario is addressed or clearly characterized.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.