dapr / dapr/components-contrib
Ability to "ResetOffset" for Kafka pub/sub
- Dominant language
- Go
- Stars
- 602
- Forks
- 580
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
## In what area(s)?
/area components (kafka pub/sub)
## Describe the feature
The current kafka pub/sub component targets just the minimum interfaces to publish and subscribe. We need an enhancement to the kafka component to support "ResetOffets". This means, we need to be able to invoke an API call to the dapr sidecar to "reset" the current offset to a previous value so that messages can be replayed.
As a side note, when dapr calls our services on an output binding for delivering messages, include the "offset" number so that clients can keep track of Message-Id to offset mapping.
**UPDATE**
The solution would involve handling metadata from PubSub subscriber's response and let each component implement custom handling of metadata. So, the subscriber can respond with a custom metadata to enable these scenarios.
Contributor guide
Research direction
Start by examining the Kafka pub/sub component and the Dapr sidecar pub/sub API described in the issue, then trace how subscriber responses currently carry metadata. Clarify the component-specific metadata contract and how an API call would select a prior offset. Done means the design is agreed and both offset replay and delivery of offset metadata are supported and tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100