[Feature] kafka cdc support metadata_column
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Get metadata from kafka header, such as database, tablename, and other keys/values
flink cdc kafka pipeline writes database, tablename to kafka header, plan to support custom key/value
https://github.com/apache/flink-cdc/pull/2938
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
No repository files or tests are named. Start by tracing the Kafka CDC pipeline and reading the linked Flink CDC PR to understand the database and table headers; done should support a metadata_column that reads those headers and custom key/value metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100