confluentinc / confluentinc/confluent-kafka-javascript
uncommittedOffsets : Clarification and Alternatives
- Dominant language
- TypeScript
- Stars
- 304
- Forks
- 45
- Avg merge
- 11h 47m
- Merged PRs (30d)
- 5
Description
Hi Team,
We are migrating to Javascript client from KafkaJS and using APIs under:
`const { Kafka } = require("@confluentinc/kafka-javascript").KafkaJS;`
We would like to get clarification on uncommittedOffsets support in the migration documentation present here:
**Within the eachBatch callback, use of uncommittedOffsets is unsupported**
[Link to documentation](https://docs.confluent.io/kafka-clients/javascript/current/migration.html#javascript-migrate-from-kafkajs:~:text=Within%20the%20eachBatch%20callback%2C%20use%20of%20uncommittedOffsets%20is%20unsupported%2C)
Is there any alternative provided to track and commit offsets with a similar fine-tuned control? I am attaching some example usages below.
Example Usage 1:
```
await resolveOffsets(batch.messages, resolveOffset)
await commitOffsetsIfNecessary(uncommittedOffsets())
```
Usage 2: Where we track and update uncommittedOffsets
```
const uncommittedOffsets = []
....
consumer.uncommittedOffsets = consumer.uncommittedOffsets.concat(uncommittedOffsets)
....
if (consumer.uncommittedOffsets.length === 0) {
return done(null, false)
}
consumer.commitOffsetsIfNecessary(consumer.uncommittedOffsets)
.then(() => {
consumer.uncommittedOffsets = []
return done(null, true)
})
.catch(err => {
return done(err, false)
})
}
```
We do not want to use `autocommit :true` since, it does not provide the finer-grained control over when and how offsets are committed. So, please let us know if there's any alternative.
Thanks in advance.
Contributor guide
Research direction
Start with the migration documentation section on uncommittedOffsets and compare the eachBatch examples with the two KafkaJS usage patterns in the issue. Determine whether the client exposes a supported equivalent for fine-grained offset tracking and committing; done means the migration guidance clearly answers the question or identifies the supported limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- stream-processing
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100