apache / apache/paimon

[Bug] When using mysql-CDC Database synchronization Action On Combined Mode, mysql field changes cause tasks to wait indefinitely

Open
#4,324 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

master : 66cc214b

### Compute Engine

flink1.18

### Minimal reproduce step

1、run mysql-cdc database action combined mode:
'args'=' mysql_sync_database
--warehouse hdfs:///paimon/warehouse
--database ods_hobby_content
--mysql_conf hostname=***
--mysql_conf username=***
--mysql_conf password=***
--mysql_conf database-name=***
--mysql_conf server-id=10011-10021
--table_prefix ods_
--table_conf changelog-producer=input
--type_mapping to-nullable,tinyint1-not-bool
--table_conf bucket=3
--table_conf sink.parallelism=3
--table_conf sink.use-managed-memory-allocator=true
--table_conf deletion-vectors.enabled=true
--table_conf write-buffer-size=512m
--table_conf target-file-size=256m
--table_conf sink.managed.writer-buffer-memory=512m
--table_conf write-buffer-spillable=true
--table_conf sort-spill-threshold=10
--table_conf num-sorted-run.stop-trigger=2147483647
--table_conf lookup-wait=false
--mode combined

2、After running for a period of time, when the mysql table field changes, the flink log keeps output:
Field " + filedName + " not found. Waiting for schema update.
Field " + filedName + " not found. Waiting for schema update.
Field " + filedName + " not found. Waiting for schema update.

And it will continue until the flink job dies

I located the source code output location of the log:
org.apache.paimon.flink.sink.cdc.CdcRecordUtils#toGenericRow(:92line)
![image](https://github.com/user-attachments/assets/679dcb59-e433-4e53-9dae-24044c9358fd)

It seems that the field has been waiting for a change, and the data flow that handles the schema change has not been processed properly

Wonder if it is possible that the current topology never has time to process field changes when the data synchronization is being compressed, or that the schema change operator needs to add state?

### What doesn't meet your expectations?
no error

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at org.apache.paimon.flink.sink.cdc.CdcRecordUtils#toGenericRow, especially the field-not-found waiting path, then trace how schema changes move through the mysql-CDC database action in combined mode. Reproduce with the supplied Flink 1.18 configuration and a MySQL field change; done means the job processes the updated schema without waiting indefinitely or dying.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.