open-telemetry / open-telemetry/opentelemetry-cpp-contrib

coredump when openresty ngx.location.capture is call

Open
#134 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
153
Forks
184
Avg merge
2d 9h
Merged PRs (30d)
26

Description

Describe your environment
Mac docker run openresty/openresty:alpine

Steps to reproduce
1.
compile the otel so into image
Dockerfile content

FROM openresty/openresty:alpine AS builder
ENV HTTP_PROXY=http://host.docker.internal:7890
ENV HTTPS_PROXY=http://host.docker.internal:7890
ENV OPENTELEMETRY_VERSION v1.2.0
RUN echo 'https://mirrors.ustc.edu.cn/alpine/v3.15/main' > /etc/apk/repositories \
    && echo 'https://mirrors.ustc.edu.cn/alpine/v3.15/community' >> /etc/apk/repositories\
    && apk update \
    && apk add --update git alpine-sdk build-base cmake linux-headers libressl-dev pcre-dev zlib-dev grpc-dev curl-dev protobuf-dev c-ares-dev re2-dev
RUN git clone --shallow-submodules --depth 1 --recurse-submodules -b ${OPENTELEMETRY_VERSION} \
      https://github.com/open-telemetry/opentelemetry-cpp.git \
      && cd opentelemetry-cpp \
      && mkdir build \
      && cd build \
      && cmake -DCMAKE_BUILD_TYPE=Debug \
        -DCMAKE_INSTALL_PREFIX=/install \
        -DCMAKE_PREFIX_PATH=/install \
        -DWITH_OTLP=ON \
        -DWITH_OTLP_GRPC=ON \
        -DWITH_OTLP_HTTP=OFF \
        -DBUILD_TESTING=OFF \
        -DWITH_EXAMPLES=OFF \
        -DWITH_ABSEIL=ON \
        -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
        .. \
      && make -j2 \
      && make install
RUN git clone https://github.com/open-telemetry/opentelemetry-cpp-contrib.git \
      && cd opentelemetry-cpp-contrib/instrumentation/nginx \
      && mkdir build \
      && cd build \
      && cmake -DCMAKE_BUILD_TYPE=Debug \
        -DNGINX_BIN=/usr/local/openresty/bin/openresty \
        -DCMAKE_PREFIX_PATH=/install \
        -DCMAKE_INSTALL_PREFIX=/usr/local/openresty/nginx/modules \
        -DCURL_LIBRARY=/usr/lib/libcurl.so.4 \
        .. \
      && make -j2 \
      && make install

FROM openresty/openresty:alpine

RUN echo 'https://mirrors.ustc.edu.cn/alpine/v3.15/main' > /etc/apk/repositories \
    && echo 'https://mirrors.ustc.edu.cn/alpine/v3.15/community' >> /etc/apk/repositories\
    && apk update \
    && apk add --update alpine-sdk build-base cmake linux-headers libressl-dev pcre-dev zlib-dev grpc-dev curl-dev protobuf-dev c-ares-dev re2-dev

COPY --from=builder /usr/local/openresty/nginx/modules/otel_ngx_module.so /usr/local/openresty/nginx/modules/otel_ngx_module.so

WORKDIR /usr/local/openresty

ENV PATH=$PATH:/usr/local/openresty/luajit/bin:/usr/local/openresty/nginx/sbin:/usr/local/openresty/bin

# Copy nginx configuration files

EXPOSE 9080 9443 80 8080

CMD ["/usr/local/openresty/bin/openresty", "-g", "daemon off;"]

STOPSIGNAL SIGQUIT

  1. docker build -t openresty-otel:v1 .
  2. docker run -it --privileged openresty-otel:v1
  3. go into the container and change nginx.conf
load_module /usr/local/openresty/nginx/modules/otel_ngx_module.so;

and then add location

location = /test1 {
        access_by_lua '
                ngx.log(ngx.ERR, "========== go here ===============")
                ngx.print("this is lua")
                ngx.print(ngx.var.uri)
        ';
    }

    location = /test2 {
        access_by_lua '
        local key = ngx.var.http_user_agent
        local res = ngx.location.capture(
                "/test1", { args = { key = key } }
        )
        ngx.print(res.body)
        ';
    }

  1. openresty -s reload
  2. curl 127.0.0.1/test2 -iv , coredump

What is the expected behavior?
output normaly

What is the actual behavior?
2022/03/26 06:02:35 [alert] 1#1: worker process 229 exited on signal 11
2022/03/26 06:02:59 [alert] 1#1: worker process 234 exited on signal 11
2022/03/26 06:03:54 [alert] 1#1: worker process 242 exited on signal 11

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Use the supplied Dockerfile to build the image and load otel_ngx_module.so, then reproduce the failure with the nginx.conf locations using ngx.location.capture and curl /test2. Start by isolating the crash around the OpenResty request flow; done means /test2 returns the expected output without worker processes exiting on signal 11.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, lua, nginx
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.