apache / apache/pulsar-client-node

Allow BigInt/Buffer to be used as initialSequenceId and/or sequenceId

Open
#124 0 comments 0 reactions 0 assignees View on GitHub
triage/week-40 type/feature
Dominant language
C++
Stars
164
Forks
98
PR merge metrics
No merged PRs in 30d

Description

Hello there,

Right now the producer's sequence ID is limited to 4 bytes here:

- https://github.com/apache/pulsar-client-node/blob/master/src/ProducerConfig.cc#L76
- https://github.com/apache/pulsar-client-node/blob/master/src/Message.cc#L174

However we could allow 8 bytes integers if we allow producer/message to receive a [BigInt](https://nodejs.org/api/n-api.html#n_api_napi_get_value_bigint_int64) and/or a [8 bytes buffer](https://nodejs.org/api/buffer.html#buffer_buf_writebigint64le_value_offset).

I'm trying to match a dataset that uses 8 bytes integers for keys as my sequence ID, that's why I found this limitation :-)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the sequence ID handling at src/ProducerConfig.cc#L76 and src/Message.cc#L174, along with the producer and message interfaces that use those values. Determine how BigInt and 8-byte Buffer inputs should be accepted, then verify that 8-byte sequence IDs can be passed through without truncation and that the existing 4-byte behavior remains supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, javascript, node.js
Domain
api, backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
43/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.