apache / apache/paimon

[Bug] s3://paas-flink-prod/.../bucket-0/data-5da975ee-318e-4ba4-b3f7-ad112dae5247-0.parquet is not a Parquet file. Expected magic number at tail, but found [21, 0, 21, -32]

Open
#5,081 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

paimon-flink-1.20-1.0.1.jar
paimon-s3-1.0.1.jar
paimon-flink-action-1.0.1.jar

### Compute Engine

flink-1.20.0

### Minimal reproduce step

Use mysql cdc sync table to paimon table which on s3. it cannot complet checkpoint, taskmanager report:

```
Caused by: java.lang.RuntimeException: s3://paas-flink-prod/flink-paimon/wh/chen.db/department/bucket-0/data-65dbb220-7017-468d-affb-1de9dd6e4105-0.parquet is not a Parquet file. Expected magic number at tail, but found [21, 0, 21, -32]
at org.apache.paimon.shade.org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:162) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.shade.org.apache.parquet.hadoop.ParquetFileReader.(ParquetFileReader.java:243) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.format.parquet.ParquetUtil.getParquetReader(ParquetUtil.java:85) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.format.parquet.ParquetUtil.extractColumnStats(ParquetUtil.java:52) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.format.parquet.ParquetSimpleStatsExtractor.extractWithFileInfo(ParquetSimpleStatsExtractor.java:78) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.format.parquet.ParquetSimpleStatsExtractor.extract(ParquetSimpleStatsExtractor.java:71) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.io.StatsCollectingSingleFileWriter.fieldStats(StatsCollectingSingleFileWriter.java:105) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.io.KeyValueDataFileWriter.result(KeyValueDataFileWriter.java:169) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.io.KeyValueDataFileWriter.result(KeyValueDataFileWriter.java:58) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.io.RollingFileWriter.closeCurrentWriter(RollingFileWriter.java:135) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.io.RollingFileWriter.close(RollingFileWriter.java:167) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.mergetree.MergeTreeWriter.flushWriteBuffer(MergeTreeWriter.java:235) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
at org.apache.paimon.mergetree.MergeTreeWriter.prepareCommit(MergeTreeWriter.java:264) ~[paimon-flink-1.20-1.0.1.jar:1.0.1]
```

I have downloaded this parquet and checked it is ok.

cdc params:

local:///opt/flink/usrlib/paimon-flink-action-1.0.1.jar \
mysql_sync_table \
--warehouse s3://paas-flink-prod/flink-paimon/wh \
--database chen \
--table department \
--mysql_conf hostname=rm-xxx.mysql.rds.aliyuncs.com \
--mysql_conf username=** \
--mysql_conf password='**' \
--mysql_conf database-name='xxx' \
--mysql_conf table-name='department'

### What doesn't meet your expectations?

it's cannot use s3 as paimon warehouse backend storage, hdfs is ok.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the stack-trace entry points in ParquetUtil, ParquetSimpleStatsExtractor, StatsCollectingSingleFileWriter, and RollingFileWriter, then compare the S3 and HDFS paths used by the MySQL CDC sync. Reproduce the checkpoint failure with the stated Paimon and Flink versions and verify that S3-backed files remain valid Parquet files and checkpoints complete successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java, mysql
Domain
data-engineering, databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.