debezium / debezium/dbz

CloudEvents: Propagate "partitionkey" [DBZ-4818]

Open
#588 0 comments 0 reactions 0 assignees View on GitHub
component/core-library component/debezium-server migrated-from-jira type/task
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-4818](https://issues.redhat.com/browse/DBZ-4818)

As per the CE [partitioning extension|https://github.com/cloudevents/spec/blob/v1.0.1/extensions/partitioning.md], the "partitionkey" attribute "can be used to determine which bucket each event goes into". We should set it to a value derived from the change event key. I can think of two ways:
* Stringified JSON-representation of the key, e.g. {{"\{\"id\" : 1234, \"sub-id\" : 56}"}}
* semicolon-separated concatention of all key field(s), e.g. {{{}"id:1234;sub-id:56"{}}}, similar to what we do for the "id" field

I'd vote for the latter. Any thoughts?

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files, tests, or entry points. Start with the linked CloudEvents partitioning extension and locate the code that maps Debezium change-event keys to CloudEvents attributes. Done requires an agreed partitionkey representation and verification for the proposed key formats.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.