apache / apache/parquet-java

The error message "is not a Parquet file" doesn't show the Parquet path anymore

Open
#3,354 1 comment 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Version 1.13.1, used by Spark 3.5: https://github.com/apache/parquet-java/blob/db4183109d5b734ec5930d870cdae161e408ddba/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java#L548

used to show the parquet file path: `Cause: java.lang.RuntimeException: file:/tmp/spark-37..../00000000000000000000.checkpoint.parquet is not a Parquet file (length is too low: 10)`

In version 1.15.2, used by Spark 4: https://github.com/apache/parquet-java/blob/859eac165b08f927fa14590c33bc5f476405fb68/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java#L584

The error message says: `Cause: java.lang.RuntimeException: io.delta.kernel.defaults.internal.parquet.ParquetIOUtils$1@63d4ffeb is not a Parquet file (length is too low: 10)`

### Component(s)

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.