pulsar: support debezium protocol
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 56
- Forks
- 63
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
Is your feature request related to a problem?
The Pulsar sink only accepts canal-json as the message protocol. Users who consume TiCDC change events via Pulsar with Debezium-compatible downstream systems (e.g. Flink CDC, Debezium connectors) have no way to produce standard Debezium-format messages, even though the Debezium codec is already implemented and used by the Kafka sink.
Describe the feature you'd like
Allow protocol=debezium to be set in the Pulsar sink URI:
pulsar://host:6650/topic?protocol=debezium
This requires extending IsPulsarSupportedProtocols() in pkg/config/sink_protocol.go to include ProtocolDebezium. No new codec logic is needed since the Debezium encoder/decoder in pkg/sink/codec/debezium/ is already shared with the Kafka sink via the common codec builder.
Describe alternatives you've considered
Users currently must use canal-json and implement a custom transform layer in their consumer to convert to Debezium format. This adds operational overhead and is error-prone.
Teachability, Documentation, Adoption, Migration Strategy
The Pulsar sink URI parameter documentation should be updated to list debezium as a valid value for the protocol parameter alongside canal-json. No migration is required for existing changefeeds since the default protocol is unchanged.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pkg/config/sink_protocol.go at IsPulsarSupportedProtocols(), then inspect the shared codec handling under pkg/sink/codec/debezium/. Update the Pulsar sink protocol options so protocol=debezium is accepted, and update the Pulsar sink URI documentation to list it alongside canal-json; existing defaults should remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100