Blizzard / Blizzard/node-rdkafka
Buffer header values converted to strings
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: CentOS 7
- Node Version [e.g. 8.2.1]: 16.15.0
- NPM Version [e.g. 5.4.2]: 8.5.5
- C++ Toolchain [e.g. Visual Studio, llvm, g++]: g++
- node-rdkafka version [e.g. 2.3.3]: 2.15.0
**Steps to Reproduce**
Run attached test script. It will crash, trying to do a `readInt32LE()` operation on the Buffer due to it being previously converted to a string value.
**node-rdkafka Configuration Settings**
see script
**Additional context**
The issue is resolved when applying PR #968. Sample file to reproduce: [test.js.txt](https://github.com/Blizzard/node-rdkafka/files/10933331/test.js.txt)
**Output from 2.15.0 module**
```consumer ready. Waiting for rebalance_cb...
rebalance_cb called
running producer...
data.value: test-payload-value
[ { myID: } ]
node:internal/buffer:84
throw new ERR_BUFFER_OUT_OF_BOUNDS();
^
RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: Attempt to access memory outside buffer bounds
at new NodeError (node:internal/errors:372:5)
at boundsError (node:internal/buffer:84:11)
at Uint8Array.readInt32LE (node:internal/buffer:390:5)
at KafkaConsumer. (test.js:38:39)
at KafkaConsumer.emit (node:events:527:28)
at readCallback (./node_modules/node-rdkafka/lib/kafka-consumer.js:448:12) {
code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}
```
**Output from patched module**
```consumer ready. Waiting for rebalance_cb...
rebalance_cb called
running producer...
data.value: test-payload-value
[ { myID: } ]
myIDVal should be 579: 579
myIDVal type should be 'number': number
```
Contributor guide
Research direction
Start with the attached test.js and the stack trace at node_modules/node-rdkafka/lib/kafka-consumer.js:448, then compare the behavior with PR #968. Done means the script no longer converts header Buffer values to strings: myID should be read as 579 and report type number.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, kafka, node.js
- Domain
- backend, stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100