[Bug] Infinite loop after using online schema change to add a new column and update values
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
0.7.0
### Compute Engine
Flink 1.17.2
### Minimal reproduce step
I do this all in the local environment, please let me know if i missed some key steps, many thanks!
1. Docker build percona mysql 8.0
`docker run -d --name ps -e MYSQL_ROOT_USER=root -e MYSQL_ROOT_PASSWORD=123456 -p 3306:3306 percona/percona-server:8.0`
2. Add sample database `inventory` and table `customer_percona` with values
3. Install percona-toolkit 3.0.0
4. Setup and Run Flink job
`bin/flink run lib/paimon-flink-action-0.7-SNAPSHOT.jar mysql_sync_database --warehouse hdfs://localhost:8020/paimon/warehouse --database inventory --mysql_conf hostname=127.0.0.1 --mysql_conf port=3306 --mysql_conf username=root --mysql_conf password=123456 --mysql_conf database-name=inventory --mysql_conf server-time-zone=UTC --catalog_conf metastore=filesystem --table_conf bucket=4 --table_conf changelog-producer=input --table_conf sink.parallelism=4 --type_mapping to-nullable`
5. Proceed online schema change to add new column
`pt-online-schema-change --alter "add column bag_fee double" h=127.0.0.1,p=123456,u=root,P=3306,D=inventory,t=customer_percona --execute`
6. Update value in the new created column
``` UPDATE `inventory`.`customer_percona` SET `bag_fee` = '0' WHERE (`id` = '1');```
### What doesn't meet your expectations?
The writer keeps showing `Busy max 100%`
The flink-root-taskexecutor-xxxx.log keep waiting for schema update. `2024-05-31 12:25:45,596 INFO org.apache.paimon.flink.sink.cdc.CdcRecordUtils [] - Field bag_fee not found. Waiting for schema update.`
---
Before

---
Online schema change

---
After updated value in the new created column


### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the Flink job from the issue using the listed MySQL and percona-toolkit commands. Inspect the CdcRecordUtils path that logs “Field bag_fee not found. Waiting for schema update.” Done means the online schema change and subsequent update no longer leave the writer at Busy max 100% or waiting indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100