open-telemetry / open-telemetry/opentelemetry-cpp-contrib

Nginx 1.22.0 doesn't start with OpenTelemetry enabled

Open
#199 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
153
Forks
184
Avg merge
2d 9h
Merged PRs (30d)
26

Description

Describe your environment
The issue was detected when using:

  • nginx:1.22.0-alpine
  • opentelemetry-cpp v.1.2.0
  • latest version of the nginx instrumentation

Steps to reproduce
It is possible to reproduce using the following Dockerfile and nginx.conf sample:

FROM nginx:1.22.0-alpine as builder

ENV OPENTELEMETRY_VERSION v1.2.0

RUN apk update \
  && apk add --update \
      alpine-sdk build-base cmake linux-headers libressl-dev pcre-dev zlib-dev \
      grpc grpc-dev curl-dev protobuf-dev c-ares-dev \
      re2-dev abseil-cpp

RUN git clone --shallow-submodules --depth 1 --recurse-submodules -b ${OPENTELEMETRY_VERSION} \
  https://github.com/open-telemetry/opentelemetry-cpp.git \
  && cd opentelemetry-cpp \
  && mkdir build \
  && cd build \
  && cmake -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=/install \
    -DCMAKE_PREFIX_PATH=/install \
    -DWITH_OTLP=ON \
    -DWITH_OTLP_GRPC=ON \
    -DWITH_OTLP_HTTP=OFF \
    -DBUILD_TESTING=OFF \
    -DWITH_EXAMPLES=OFF \
    -DWITH_ABSEIL=ON \
    -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    .. \
  && make -j2 \
  && make install

RUN git clone https://github.com/open-telemetry/opentelemetry-cpp-contrib.git \
  && cd opentelemetry-cpp-contrib/instrumentation/nginx \
  && mkdir build \
  && cd build \
  && cmake -DCMAKE_BUILD_TYPE=Release \
    -DNGINX_BIN=/usr/sbin/nginx \
    -DCMAKE_PREFIX_PATH=/install \
    -DCMAKE_INSTALL_PREFIX=/usr/lib/nginx/modules \
    -DCURL_LIBRARY=/usr/lib/libcurl.so.4 \
    .. \
  && make -j2 \
  && make install

FROM nginx:1.22.0-alpine

RUN apk update \
  && apk add --update \
      grpc protobuf c-ares
      # gdb # uncomment this if you need to debug nginx

COPY --from=builder /usr/lib/nginx/modules/otel_ngx_module.so /usr/lib/nginx/modules/

COPY nginx/conf/nginx.conf /etc/nginx/nginx.conf

# CMD ["/usr/sbin/nginx-debug", "-g", "daemon off;"]

worker_processes  auto;
error_log  /dev/stderr;
pid        /tmp/nginx.pid;
worker_rlimit_nofile 8192;
worker_rlimit_core  500M;
working_directory /tmp;
debug_points abort;
load_module /usr/lib/nginx/modules/otel_ngx_module.so;

events {
  worker_connections  4096;  ## Default: 1024
}

http {
  server {
    listen 8080;

    location /status {
      # opentelemetry off;
      stub_status on;
      access_log  off;
      allow 127.0.0.1;
      deny all;
    }

    location / {
      root   html;
      index  index.html index.htm;
    }
  }
}

What is the expected behavior?
Nginx should start normally

What is the actual behavior?
Nginx logs the following messages and fails:

/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2022/07/28 09:16:02 [emerg] 1#1: dlopen() "/usr/lib/nginx/modules/otel_ngx_module.so" failed (Error relocating /usr/lib/nginx/modules/otel_ngx_module.so: _ZN4absl12lts_2021110216variant_internal21ThrowBadVariantAccessEv: symbol not found) in /etc/nginx/nginx.conf:8
nginx: [emerg] dlopen() "/usr/lib/nginx/modules/otel_ngx_module.so" failed (Error relocating /usr/lib/nginx/modules/otel_ngx_module.so: _ZN4absl12lts_2021110216variant_internal21ThrowBadVariantAccessEv: symbol not found) in /etc/nginx/nginx.conf:8

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Build the provided Dockerfile and load the module with the supplied nginx.conf to reproduce the startup failure. Start from the dlopen() relocation error for otel_ngx_module.so and compare the builder and runtime dependencies; done means nginx starts normally with OpenTelemetry enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, nginx
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.