Blizzard / Blizzard/node-rdkafka

Getting npm-gyp errors regarding node-headers when trying to install node-rdkafka on a docker container

Open
#1,011 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

For some reason I get that when npm intall tries to install node-rdkafka on a container then it cannot download the headers:

> #0 16.21 npm ERR! gyp http GET https://unofficial-builds.nodejs.org/download/release/v18.16.0/node-v18.16.0-headers.tar.gz

When clicking on the link i could download the package no problem.

### Chatgpt remarks when trying to see what kind of error it is:

> This error message appears to be related to an issue with the Node.js package manager (npm) while trying to install or build a package. Specifically, it suggests that there was a problem with fetching the required header files for Node.js version 18.16.0, which are necessary for building native addons.
>
> The error message indicates that the request to download the headers file failed due to a network error, with the reason being "getaddrinfo EAI_AGAIN unofficial-builds.nodejs.org". This suggests that the npm tool was unable to resolve the domain name "unofficial-builds.nodejs.org" to an IP address, which could be due to a DNS resolution issue or a problem with the network connection.
>
> To resolve this issue, you could try running the npm command again after ensuring that your network connection is stable and there are no DNS resolution issues. If the problem persists, you could try manually downloading the required headers file from the Node.js website and placing it in the appropriate location for the npm tool to use.
>

### Command used:
`docker compose up user-info-manager`

### Result of command:
```
[+] Running 1/1
! user-info-manager Warning 13.0s
[+] Building 46.1s (10/18)
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 848B 0.1s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 35B 0.1s
=> [internal] load metadata for docker.io/library/node:18-alpine 3.3s
=> [ 1/14] FROM docker.io/library/node:18-alpine@sha256:1ccc70acda680aa4ba47f53e7c40b2d4d6892de74817128e0662d32647dd7f4d 0.1s
=> => resolve docker.io/library/node:18-alpine@sha256:1ccc70acda680aa4ba47f53e7c40b2d4d6892de74817128e0662d32647dd7f4d 0.1s
=> [internal] load build context 0.2s
=> => transferring context: 966B 0.2s
=> CACHED [ 2/14] RUN apk update 0.0s
=> [ 3/14] RUN apk --no-cache add bash ca-certificates lz4-dev musl-dev cyrus-sasl-dev openssl-dev librdkafka-dev=1.9.2-r0 make g++ python3 25.2s
=> [ 4/14] WORKDIR /usr/src/app/user-info-manager/ 0.2s
=> [ 5/14] COPY package*.json . 0.2s
=> ERROR [ 6/14] RUN npm install 16.9s
------
> [ 6/14] RUN npm install:
#0 7.253 npm WARN deprecated keycloak-connect@21.1.0: This package is deprecated and will be removed in the future. We will shortly provide more details on removal date, and recommended alternatives.
#0 16.21 npm notice
#0 16.21 npm notice New minor version of npm available! 9.5.1 -> 9.6.6
#0 16.21 npm notice Changelog:
#0 16.21 npm notice Run `npm install -g npm@9.6.6` to update!
#0 16.21 npm notice
#0 16.21 npm ERR! code 1
#0 16.21 npm ERR! path /usr/src/app/user-info-manager/node_modules/node-rdkafka
#0 16.21 npm ERR! command failed
#0 16.21 npm ERR! command sh -c node-gyp rebuild
#0 16.21 npm ERR! gyp info it worked if it ends with ok
#0 16.21 npm ERR! gyp info using node-gyp@9.3.1
#0 16.21 npm ERR! gyp info using node@18.16.0 | linux | x64
#0 16.21 npm ERR! gyp info find Python using Python version 3.10.11 found at "/usr/bin/python3"
#0 16.21 npm ERR! gyp http GET https://unofficial-builds.nodejs.org/download/release/v18.16.0/node-v18.16.0-headers.tar.gz
#0 16.21 npm ERR! gyp WARN install got an error, rolling back install
#0 16.21 npm ERR! gyp ERR! configure error
#0 16.21 npm ERR! gyp ERR! stack FetchError: request to https://unofficial-builds.nodejs.org/download/release/v18.16.0/node-v18.16.0-headers.tar.gz failed, reason: getaddrinfo EAI_AGAIN unofficial-builds.nodejs.org
#0 16.21 npm ERR! gyp ERR! stack at ClientRequest. (/usr/local/lib/node_modules/npm/node_modules/node-gyp/node_modules/minipass-fetch/lib/index.js:130:14)
#0 16.21 npm ERR! gyp ERR! stack at ClientRequest.emit (node:events:513:28)
#0 16.21 npm ERR! gyp ERR! stack at TLSSocket.socketErrorListener (node:_http_client:502:9)
#0 16.21 npm ERR! gyp ERR! stack at TLSSocket.emit (node:events:525:35)
#0 16.21 npm ERR! gyp ERR! stack at emitErrorNT (node:internal/streams/destroy:151:8)
#0 16.21 npm ERR! gyp ERR! stack at emitErrorCloseNT (node:internal/streams/destroy:116:3)
#0 16.21 npm ERR! gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
#0 16.21 npm ERR! gyp ERR! System Linux 5.15.90.1-microsoft-standard-WSL2
#0 16.21 npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
#0 16.21 npm ERR! gyp ERR! cwd /usr/src/app/user-info-manager/node_modules/node-rdkafka
#0 16.21 npm ERR! gyp ERR! node -v v18.16.0
#0 16.21 npm ERR! gyp ERR! node-gyp -v v9.3.1
#0 16.21 npm ERR! gyp ERR! not ok
#0 16.21
#0 16.21 npm ERR! A complete log of this run can be found in:
#0 16.21 npm ERR! /root/.npm/_logs/2023-05-07T12_01_15_753Z-debug-0.log
------
failed to solve: executor failed running [/bin/sh -c npm install]: exit code: 1
```

### docker-compose.yaml:
```
version: "3.0"
services:
user-info-manager:
build:
context: ./user-info-manager
dockerfile: Dockerfile
image: userinfomanager
container_name: userinfomanager
ports:
- "5000:${APP_PORT}"
env_file:
- .env
depends_on:
- broker
- zookeeper
- keycloak-ready
restart: unless-stopped
keycloak-ready:
image: gmitis/keycloak:latest
container_name: keycloak-ready
environment:
- KEYCLOAK_ADMIN=admin
- KEYCLOAK_ADMIN_PASSWORD=admin
ports:
- "8080:8080"
command: start-dev
restart: unless-stopped

zookeeper:
image: confluentinc/cp-zookeeper:7.3.2
container_name: zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
restart: unless-stopped

broker:
image: confluentinc/cp-kafka:7.3.2
container_name: broker
ports:
- "9092:9092"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
restart: unless-stopped
```

### Dockerfile:
```
FROM node:18-alpine
RUN apk update
RUN apk --no-cache add \
bash \
ca-certificates \
lz4-dev \
musl-dev \
cyrus-sasl-dev \
openssl-dev \
librdkafka-dev=1.9.2-r0 \
make \
g++ \
python3

WORKDIR /usr/src/app/user-info-manager/
COPY package*.json .
RUN npm install

WORKDIR /usr/src/app/choreographer/
COPY ./kafka/package*.json .
RUN npm install

WORKDIR /usr/src/app/
COPY . ./user-info-manager/
COPY ./kafka ./choreographer/kafka/
COPY ./models/EventType* ./choreographer/models/

WORKDIR /usr/src/app/user-info-manager/
ENV NODE_ENV=production
EXPOSE $APP_PORT
CMD ["npm", "start"]
```

# Solution:
Turns out the problem was created by the TPLink wifi extender than i had. If you connect to the original router you are going to be fine.

Contributor guide

Open the contributing guide

Research direction

Start with the Dockerfile and the user-info-manager package manifests; reproduce with docker compose up user-info-manager and inspect the node-rdkafka node-gyp header-download failure. Done means npm install completes in the documented container setup, with the cause and reproducibility confirmed.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, javascript, nodejs
Domain
build-system, devops
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.