apache / apache/hudi

[bug] Repeating hudi_table_changes query on the same large table gets stuck

Open
#10,096 3 comments 0 reactions 0 assignees View on GitHub
area:incr-processing area:sql priority:high
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 4h
Merged PRs (30d)
112

Description

**Describe the problem you faced**

**To Reproduce**

Steps to reproduce the behavior:

1.
spark-sql --packages org.apache.hudi:hudi-spark3.2-bundle_2.12:0.14.0 --master yarn --driver-memory 8g --num-executors 10 --conf spark.dynamicAllocation.maxExecutors=20 --executor-memory 4G --executor-cores 2 --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --conf spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog --conf spark.kryo.registrator=org.apache.spark.HoodieSparkKryoRegistrar --conf spark.sql.autoBroadcastJoinThreshold=2G --conf spark.memory.storageFraction=0.5 --conf spark.sql.broadcastTimeout=60000 --conf spark.yarn.priority=5 --conf spark.sql.broadcastTimeout=600000 --conf spark.network.timeout=600000s --conf spark.eventLog.enable=false --conf spark.driver.maxResultSize=4g --conf spark.driver.extraJavaOptions=-XX:-UseGCOverheadLimit --conf spark.executor.extraJavaOptions=-XX:-UseGCOverheadLimit --name zyc_test --conf spark.dynamicAllocation.enabled=false

2.SELECT count(1) FROM hudi_table_changes('bi_ods_real.ods_log_smart_datapoint_report_batch_rt', 'latest_state', '20231114033500000', '20231114040500000');

Results can be returned normally
3. SELECT count(1) FROM hudi_table_changes('bi_ods_real.ods_log_smart_datapoint_report_batch_rt', 'latest_state', '20231114033500000', '20231114040500000');

It's stuck and won't exit. It's hard to exit by Ctrl+C.

![image](https://github.com/apache/hudi/assets/15028279/4d1be4a4-7cf7-48f2-8484-d56ddf58187f)

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment Description**

* Hudi version :0.14.0

* Spark version :3.2.1

* Hive version :3.1.3

* Hadoop version :3.2.2

* Storage (HDFS/S3/GCS..) :s3

* Running on Docker? (yes/no) :no

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue with the supplied spark-sql configuration and the two identical hudi_table_changes queries against the S3-backed table. Trace the table-changes query path to determine why the second count remains stuck; done means repeated execution returns normally instead of hanging.

Written by the indexing model from the issue text.

Assessment

Domain
cloud, data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.