Kafka Sink Connector: support non-String keys
Open
good first issue
help wanted
type/enhancement
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
Currently the Kafka Sink Connector is able to support only keys with a String value.
We should support the generic byte[] key, as it is in Kafka.
Also we should support other data types and possibly may the generic key/value pair of Kafka to a KeyValue Pulsar schema
Contributor guide
Research direction
Start at the Kafka Sink Connector entry point and trace the current handling of String keys. Determine how generic byte[] keys and other Kafka key types should be represented, including whether a Kafka key/value pair should map to a Pulsar KeyValue schema. Done means the supported key behavior and the intended mapping are implemented and verified by the connector's tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100