faced "No such file or directory " when query hudi
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
spark 3.3.2
hudi:0.13.1
kyuubi:1.7.0
Whey query the hudi tables used to get below error messages even refresh tables:
No such file or directory: s3a://xxxx/3cb781bd-4efc-4277-9b35-6d436f39ddc1-0_24-373672-1708766_20230727133636.parquet
### Affects Version(s)
1.7.0
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
```yaml
kyuubi.frontend.rest.bind.host 0.0.0.0
kyuubi.frontend.rest.bind.port 10099
kyuubi.frontend.protocols=THRIFT_BINARY,REST
kyuubi.credentials.hadoopfs.enabled=false
kyuubi.ha.enabled=true
kyuubi.ha.zookeeper.quorum=xxxx
kyuubi.ha.zookeeper.namespace=kyuubi_cluster001
spark.master=yarn
spark.submit.deployMode=cluster
#hudi
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.sql.extensions==org.apache.kyuubi.sql.KyuubiSparkSQLExtension,org.apache.spark.sql.hudi.HoodieSparkSessionExtension
spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog
spark.hive.metastore.uris=ehrift://xxxx:9083
spark.yarn.jars=xxx/spark3jars
spark.yarn.archive=xxxx/spark3_jars.zip
spark.sql.legacy.parquet.datetimeRebaseModeInRead=LEGACY
spark.sql.legacy.timeParserPolicy=LEGACY
spark.sql.session.timeZone=UTC
kyuubi.session.engine.idle.timeout=PT300M
kyuubi.session.engine.initialize.timeout=PT20M
#kyuubi
spark.sql.optimizer.insertRepartitionBeforewrite.enable=true
# AQE
spark.sql.adaptive.enabled=true
spark.sql.adaptive.advisoryPartitionSizeInBytes=64M
spark.sql.adaptive.coalescePartitions.minPartitionNum=1
# spark
spark.executor.memory=6G
spark.driver.memory=8G
spark.executor.cores=1
spark.sql.autoBroadcastJoinThreshold=20971520
spark.sql.adaptive.autoBroadcastJoinThreshold=20971520
spark.sql.broadcastTimeout=1200
spark.sql.legacy.parquet.datetimeRebaseModeInRead=LEGACY
spark.sql.legacy.timeParserPolicy=LEGACY
spark.sql.parquet.int96RebaseModeInWrite=LEGACY
spark.sql.hive.convertMetastoreParquet=true
spark.memory.fraction=0.8
spark.memory.storageFraction=0.3
# Overhead
spark.executor.memoryOverhead=2G
spark.driver.memoryOverhead=1G
spark.driver.maxResultSize=1G
#spark.memory.offHeap.enabled=true
#spark.memory.offHeap.size=2G
spark.dynamicAllocation.enabled=true
spark.shuffle.service.enabled=true
spark.dynamicAllocation.minExecutors=6
spark.dynamicAllocation.maxExecutors=60
spark.dynamicAllocation.executorAllocationRatio=0.5
spark.dynamicAllocation.sustainedSchedulerBacklogTimeout=5s
spark.dynamicAllocation.cachedExecutorIdleTimeout=30min
spark.dynamicAllocation.executorIdleTimeout=60s
# gc
spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=35 -XX:G1ReservePercent=15 -XX:+DisableExplicitGC -Duser.timezone=UTC
spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=35 -XX:G1ReservePercent=15 -XX:+DisableExplicitGC -Duser.timezone=UTC
#
```
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
Contributor guide
Research direction
Reproduce the Hudi query using Spark 3.3.2, Hudi 0.13.1, and Kyuubi 1.7.0 with the listed configurations. Start by collecting the missing Kyuubi server and engine logs and checking why the referenced s3a parquet file is unavailable; done means the cause is identified and a verified fix or configuration change is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100