apache / apache/hop

[Bug]: Error reading parquet file

Open
#3,598 4 comments 0 reactions 0 assignees View on GitHub
Hop Gui P3 Pipelines
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
19h
Merged PRs (30d)
227

Description

### Apache Hop version?

2.7.0

### Java version?

21.0.2

### Operating system

Windows

### What happened?

I'm using Parquet Input, following the documentation I started with Get File names and called Parquet Input File;
I chose the filename as Filename Field and looked for a parquet file to pull the fields, everything ok so far....but it presents the following error when trying to read the files:
### Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException : Relative path in absolute URI: ParquetStream of file 'E:%5CQSR%5CTEMP%5CSIPNI_COVID19_OUTROS_21012024_06_50.parquet'

I'm on a Windows machine and the absolute path of the file is:
### E:\QSR\TEMP\SIPNI_COVID19_OUTROS_21012024_06_50.parquet

I'm trying to read several files, and after some tests I realized that the error appeared in a single file.
I then tested searching for the fields only from this file and realized that the field types (Type) had two types that were different from the other files.
Example, in file 1 the column type was Integer and in file 2 the column was of type Number. After adjusting this and leaving everything as String, the error no longer appeared and I was able to read all the files.

I think the error message about not finding the file or about an error reading the files should be more explicit.
I received a correct error when, in the tests, the file did not exist, but when the files existed and the number and column names were the same, but the data type was different between the files, the error appeared.

![erro_hop](https://github.com/apache/hop/assets/86902585/4a777b78-7bb1-41fd-82ff-766054e3943c)

### Issue Priority

Priority: 3

### Issue Component

Component: Hop Gui, Component: Pipelines

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue in the Parquet Input component using Get File names and Parquet Input File on Windows with multiple files whose column types differ. Inspect how the component handles the Windows path and reports schema or read failures. Done means the failure is identified clearly as a file or schema mismatch and the error message distinguishes it from a missing file.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.