[SUPPORT] Flink COW upsert is too slow to sink the data to Hudi-S3
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
**Describe the problem you faced**
Flink + Hudi COW + upsert mode + S3 path -> too slow. Nearly taking half an hour for 13k records only.
Even MOR is very slow. What I have noticed is that flink is slow, while running the same process on AWS glue is comparatively fast. With same configs, it should give take same time to process
**To Reproduce**
Steps to reproduce the behavior:
Use these configuration to reproduce
" 'connector' = 'hudi'," +
" 'path' = '" + S3_PATH + "'," +
" 'table.type' = 'COPY_ON_WRITE'," +
" 'hoodie.table.name' = '" + HUDI_TABLE_NAME + "'," +
" 'hoodie.datasource.write.recordkey.field' = 'event_id'," +
" 'hoodie.datasource.write.partitionpath.field' = 'customer_uuid,year,month'," +
" 'hoodie.datasource.write.precombine.field' = 'time'," +
" 'hoodie.schema.on.read.enable' = 'true'," +
" 'hoodie.parquet.compression.codec' = 'snappy'," +
" 'hoodie.datasource.write.operation' = 'upsert',"+
" 'hoodie.parquet.max.file.size' = '10485760',"+
" 'write.tasks' = '8',"+
" 'hoodie.populate.meta.fields' = 'true',"+
" 'hoodie.table.keygenerator.class' = 'org.apache.hudi.keygen.NonpartitionedKeyGenerator',"+
" 'hoodie.metadata.enable' = 'false'"+
**Expected behavior**
It should process the data within a minutes, evne for 1 million records.
**Environment Description**
* Hudi version : 0.15.0
* Flink version : 1.18
* Hadoop version : 3.1.4
* Storage (HDFS/S3/GCS..) : S3
* Running on Docker? (yes/no) : no
**Additional context**
- use these configs
```
" 'connector' = 'hudi'," +
" 'path' = '" + S3_PATH + "'," +
" 'table.type' = 'COPY_ON_WRITE'," +
" 'hoodie.table.name' = '" + HUDI_TABLE_NAME + "'," +
" 'hoodie.datasource.write.recordkey.field' = 'event_id'," +
" 'hoodie.datasource.write.partitionpath.field' = 'customer_uuid,year,month'," +
" 'hoodie.datasource.write.precombine.field' = 'time'," +
" 'hoodie.schema.on.read.enable' = 'true'," +
" 'hoodie.parquet.compression.codec' = 'snappy'," +
" 'hoodie.datasource.write.operation' = 'upsert',"+
" 'hoodie.parquet.max.file.size' = '10485760',"+
" 'write.tasks' = '8',"+
" 'hoodie.populate.meta.fields' = 'true',"+
" 'hoodie.table.keygenerator.class' = 'org.apache.hudi.keygen.NonpartitionedKeyGenerator',"+
" 'hoodie.metadata.enable' = 'false'"+
```
- Sinking time is very random but it is taking ~half an hour for 13k and for 28K it tool 2 hours, which is very wierd

Contributor guide
No contributing guide indexed for this repository
Research direction
No source files or tests are identified. Start by reproducing the Flink 1.18 write with the listed Hudi 0.15.0 settings against S3, compare it with AWS Glue, and use the timing variation to identify the bottleneck; done means 13K and 28K records complete within the expected minutes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- data-engineering, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100