Blizzard / Blizzard/node-rdkafka

Buffer header values converted to strings

Open
#1,006 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: CentOS 7
- Node Version [e.g. 8.2.1]: 16.15.0
- NPM Version [e.g. 5.4.2]: 8.5.5
- C++ Toolchain [e.g. Visual Studio, llvm, g++]: g++
- node-rdkafka version [e.g. 2.3.3]: 2.15.0

**Steps to Reproduce**
Run attached test script. It will crash, trying to do a `readInt32LE()` operation on the Buffer due to it being previously converted to a string value.

**node-rdkafka Configuration Settings**
see script

**Additional context**
The issue is resolved when applying PR #968. Sample file to reproduce: [test.js.txt](https://github.com/Blizzard/node-rdkafka/files/10933331/test.js.txt)

**Output from 2.15.0 module**
```consumer ready. Waiting for rebalance_cb...
rebalance_cb called
running producer...
data.value: test-payload-value
[ { myID: } ]
node:internal/buffer:84
throw new ERR_BUFFER_OUT_OF_BOUNDS();
^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: Attempt to access memory outside buffer bounds
at new NodeError (node:internal/errors:372:5)
at boundsError (node:internal/buffer:84:11)
at Uint8Array.readInt32LE (node:internal/buffer:390:5)
at KafkaConsumer. (test.js:38:39)
at KafkaConsumer.emit (node:events:527:28)
at readCallback (./node_modules/node-rdkafka/lib/kafka-consumer.js:448:12) {
code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}
```

**Output from patched module**
```consumer ready. Waiting for rebalance_cb...
rebalance_cb called
running producer...
data.value: test-payload-value
[ { myID: } ]
myIDVal should be 579: 579
myIDVal type should be 'number': number
```

Contributor guide

Open the contributing guide

Research direction

Start with the attached test.js and the stack trace at node_modules/node-rdkafka/lib/kafka-consumer.js:448, then compare the behavior with PR #968. Done means the script no longer converts header Buffer values to strings: myID should be read as 579 and report type number.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kafka, node.js
Domain
backend, stream-processing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.