apache / apache/seatunnel

[Feature][MYSQL-CDC] Support canal-json format

Open
#9,779 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 17h
Merged PRs (30d)
210

Description

### Search before asking

- [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

### Description

We have a business requirement to output MySQL CDC data into a RocketMQ sink. However, we’ve noticed that most CDC sources do not support the Canal-JSON format, and only Debezium is supported. Due to business constraints, we cannot switch to Debezium.

Additionally, we’ve observed that updates are split into a DELETE and an INSERT, which complicates downstream processing.

Could you please explain:
1. Why most CDC sources do not support Canal-JSON format?
2. Are there any existing fixes or plans to add support for Canal-JSON in the future?

### Expected
```json
{
"data": [
{
"id": "111",
"name": "scooter",
"description": "Big 2-wheel scooter",
"weight": "5.18"
}
],
"database": "inventory",
"es": 1589373560000,
"id": 9,
"isDdl": false,
"mysqlType": {
"id": "INTEGER",
"name": "VARCHAR(255)",
"description": "VARCHAR(512)",
"weight": "FLOAT"
},
"old": [
{
"weight": "5.15"
}
],
"pkNames": [
"id"
],
"sql": "",
"sqlType": {
"id": 4,
"name": 12,
"description": 12,
"weight": 7
},
"table": "products",
"ts": 1589373560798,
"type": "UPDATE"
}
```
### Actually
```json
{
"data": [{
"id": 2348,
"user_id": "user_742852",
"user_name": "刘晶",
"email": "lisaanderson@example.org",
"age": 63,
"amount": 402.45,
"random_number": 109008,
"status": "inactive",
"created_at": "2025-08-25T01:56:17",
"updated_at": "2025-08-25T01:56:17"
}],
"type": "INSERT"
}
```

### Usage Scenario

_No response_

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files, tests, or entry points are identified in the issue. Start by locating the MySQL CDC serialization path and existing Debezium handling, then compare its output with the requested Canal-JSON structure; done means UPDATE events preserve the expected Canal-JSON fields and no longer appear only as separate DELETE and INSERT records.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.