apache / apache/hudi

[SUPPORT]xxx.parquet is not a Parquet file

Open
#11,178 12 comments 0 reactions 0 assignees View on GitHub
area:index issue:data-corruption priority:critical
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**

A clear and concise description of the problem.
1. When using the record_index type index to upsert an MOR type table, this exception suddenly occurred, leading to the downstream being unable to perform data reading. Other tables constructed in the same manner have not yet experienced this exception.
2. The reasons and how to fix it, as well as how to prevent it in the future, as the downstream is continuously using it, thank you very much.

**Environment Description**

* Hudi version :0.14.1

* Spark version :3.4

* Hive version :3.1.2

* Hadoop version :3.1

* Storage (HDFS/S3/GCS..) :hdfs

* Running on Docker? (yes/no) :no

**Stacktrace**

Caused by: java.lang.RuntimeException: viewfs://nbns/user/quantum_social/lakehouse/social/dwd_social_kbi_beauty_lower_v1/partition_index_date=202302/229164d5-911f-49df-91b5-cb15aecc60de-0_2531-32510-4568658_20240508183714815.parquet is not a Parquet file (length is too low: 0)
at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:540)
at org.apache.parquet.hadoop.ParquetFileReader.(ParquetFileReader.java:777)
at org.apache.parquet.hadoop.ParquetFileReader.open(ParquetFileReader.java:658)
at org.apache.spark.sql.execution.datasources.parquet.ParquetFooterReader.readFooter(ParquetFooterReader.java:53)
at org.apache.spark.sql.execution.datasources.parquet.ParquetFooterReader.readFooter(ParquetFooterReader.java:39)
at org.apache.spark.sql.execution.datasources.parquet.Spark34LegacyHoodieParquetFileFormat.footerFileMetaData$lzycompute$1(Spark34LegacyHoodieParquetFileFormat.scala:184)
at org.apache.spark.sql.execution.datasources.parquet.Spark34LegacyHoodieParquetFileFormat.footerFileMetaData$1(Spark34LegacyHoodieParquetFileFormat.scala:183)
at org.apache.spark.sql.execution.datasources.parquet.Spark34LegacyHoodieParquetFileFormat.$anonfun$buildReaderWithPartitionValues$2(Spark34LegacyHoodieParquetFileFormat.scala:187)
at org.apache.hudi.HoodieDataSourceHelper$.$anonfun$buildHoodieParquetReader$1(HoodieDataSourceHelper.scala:67)
at org.apache.hudi.HoodieBaseRelation.$anonfun$createBaseFileReader$2(HoodieBaseRelation.scala:582)
at org.apache.hudi.HoodieBaseRelation$BaseFileReader.apply(HoodieBaseRelation.scala:673)
at org.apache.hudi.RecordMergingFileIterator.(Iterators.scala:249)
at org.apache.hudi.HoodieMergeOnReadRDD.compute(HoodieMergeOnReadRDD.scala:109)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by verifying the zero-length file at the HDFS path and trace how it reaches ParquetFileReader.readFooter through Spark34LegacyHoodieParquetFileFormat and HoodieBaseRelation. Compare the record_index and Merge-On-Read context described in the report with the Hudi write path. Done means identifying the cause and documenting a verified fix and prevention guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.