Blizzard / Blizzard/node-rdkafka

Kafka Consumer + kerbos authentication +GSSAPI for node js application -> broker transport failure on Consumer end

Open
#990 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

We are trying to use Kafka Conssumer + kerbos authentication +GSSAPI for node js application for enterprise chat applictaion. We are getting below error . What could be the reason for this error , any help is highly appreciated . if you have any referennce for similar implementaion , please share ...

**broker transport failure.**

It is working fine for JAVA client , failing from NODE JS Client.

this is our client configurations.
const kafkaConsumer = new KafkaConsumer({
'client.id': ['myserver', hostname()].join('-'),
'group.id': 'kf__write_test',
'rebalance_cb': true,
'metadata.broker.list': [servers],
'security.protocol': "sasl_ssl",
'sasl.mechanisms': 'GSSAPI',
'sasl.password': 'sdadsad',
'ssl.ca.location': path.resolve(__dirname, '../../cache/certs/...'),
"sasl.kerberos.service.name": "",
"sasl.kerberos.principal": ",
"sasl.kerberos.keytab": path.resolve(__dirname, '../../cache/certs/..'),
"sasl.kerberos.kinit.cmd": "%{sasl.kerberos.keytab}",
"debug": "protocol,security,broker"
}, {
'auto.offset.reset': 'beginning',
})

**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: Mac
- Node Version [e.g. 8.2.1]: 16.10.0
- NPM Version [e.g. 5.4.2]: 7.24.0
- C++ Toolchain [e.g. Visual Studio, llvm, g++]: i belive it is llvm , please suggest how to find this.
- node-rdkafka version [e.g. 2.3.3]: 2.14.0

**Steps to Reproduce**

**node-rdkafka Configuration Settings**

this is our client configurations.
const kafkaConsumer = new KafkaConsumer({
'client.id': ['myserver', hostname()].join('-'),
'group.id': 'kf__write_test',
'rebalance_cb': true,
'metadata.broker.list': [servers],
'security.protocol': "sasl_ssl",
'sasl.mechanisms': 'GSSAPI',
'sasl.password': 'asdasda',
'ssl.ca.location': path.resolve(__dirname, '../../cache/certs/...'),
"sasl.kerberos.service.name": "",
"sasl.kerberos.principal": ",
"sasl.kerberos.keytab": path.resolve(__dirname, '../../cache/certs/..'),
"sasl.kerberos.kinit.cmd": "%{sasl.kerberos.keytab}",
"debug": "protocol,security,broker"
}, {
'auto.offset.reset': 'beginning',
})

**Error Message**

![image](https://user-images.githubusercontent.com/12228014/208814078-adb9789f-4c5d-4de4-b249-0da3a11752e6.png)

**Additional context**

![image](https://user-images.githubusercontent.com/12228014/208813938-f66e99ac-67e6-4c5d-a419-23358bc5ecf2.png)

**Error logs**
[2022-12-19T-debug.log](https://github.com/Blizzard/node-rdkafka/files/10274270/2022-12-19T-debug.log)

320 timing command:install Completed in 41507ms
321 verbose stack Error: command failed
321 verbose stack at ChildProcess. (/Users//.nvm/versions/node/v16.10.0/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
321 verbose stack at ChildProcess.emit (node:events:390:28)
321 verbose stack at maybeClose (node:internal/child_process:1064:16)
321 verbose stack at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
322 verbose pkgid node-rdkafka@2.14.0
323 verbose cwd /Users//APP/APP/chat-socket-server
324 verbose Darwin 21.6.0
325 verbose argv "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//.nvm/versions/node/v16.10.0/bin/npm" "install" "debug"
326 verbose node v16.10.0
327 verbose npm v7.24.0
328 error code 1
329 error path /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka
330 error command failed
331 error command sh -c node-gyp rebuild
332 error Debugger attached.
332 error gyp info it worked if it ends with ok
332 error gyp info using node-gyp@8.4.1
332 error gyp info using node@16.10.0 | darwin | x64
332 error gyp info find Python using Python version 3.10.9 found at "/Library/Frameworks/Python.framework/Versions/3.10/bin/python3"
332 error gyp info spawn /Library/Frameworks/Python.framework/Versions/3.10/bin/python3
332 error gyp info spawn args [
332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/gyp/gyp_main.py',
332 error gyp info spawn args 'binding.gyp',
332 error gyp info spawn args '-f',
332 error gyp info spawn args 'make',
332 error gyp info spawn args '-I',
332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/build/config.gypi',
332 error gyp info spawn args '-I',
332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/addon.gypi',
332 error gyp info spawn args '-I',
332 error gyp info spawn args '/Users//Library/Caches/node-gyp/16.10.0/include/node/common.gypi',
332 error gyp info spawn args '-Dlibrary=shared_library',
332 error gyp info spawn args '-Dvisibility=default',
332 error gyp info spawn args '-Dnode_root_dir=/Users//Library/Caches/node-gyp/16.10.0',
332 error gyp info spawn args '-Dnode_gyp_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp',
332 error gyp info spawn args '-Dnode_lib_file=/Users//Library/Caches/node-gyp/16.10.0/<(target_arch)/node.lib',
332 error gyp info spawn args '-Dmodule_root_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka',
332 error gyp info spawn args '-Dnode_engine=v8',
332 error gyp info spawn args '--depth=.',
332 error gyp info spawn args '--no-parallel',
332 error gyp info spawn args '--generator-output',
332 error gyp info spawn args 'build',
332 error gyp info spawn args '-Goutput_dir=.'
332 error gyp info spawn args ]
332 error Debugger attached.
332 error Waiting for the debugger to disconnect...
332 error gyp: Call to 'node ./util/get-env.js BUILD_LIBRDKAFKA 1' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
332 error gyp ERR! configure error
332 error gyp ERR! stack Error: gyp failed with exit code: 1
332 error gyp ERR! stack at ChildProcess.onCpExit (/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/lib/configure.js:259:16)
332 error gyp ERR! stack at ChildProcess.emit (node:events:390:28)
332 error gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
332 error gyp ERR! System Darwin 21.6.0
332 error gyp ERR! command "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/.bin/node-gyp" "rebuild"
332 error gyp ERR! cwd /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka
332 error gyp ERR! node -v v16.10.0
332 error gyp ERR! node-gyp -v v8.4.1
332 error gyp ERR! not ok
332 error Waiting for the debugger to disconnect...
333 verbose exit 1

Contributor guide

Open the contributing guide

Research direction

Start with the node-gyp output around binding.gyp and util/get-env.js, then review the supplied node-rdkafka configuration and debug log. Reproduce the installation or consumer failure on the stated macOS, Node.js, and node-rdkafka versions. Done means identifying a confirmed cause with a reproducible result or a documented resolution path.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.