apache / apache/parquet-java

ProtoReader does not iterate over the parquet file correctly.

Open
#2,497 0 comments 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

The `ProtoParquetReader` does not iterate over the parquet file correctly, but it gets stuck in the first element and keeps reading as many times as elements the file contained.

In my Scala example I am just reading from a local file that I know for sure it contains right data.

```

val hadoopCOnf = new Configuration()

val outfile: String = genTemporaryFile()

val r: ParquetReader[Event.Builder] = {
ProtoParquetReader.builder[Event.Builder](new Path(outfile)).withConf(hadoopCOnf).build()
}

```

Notice that the proto schema that I am using is generated from 

The generated proto implements com.google.protobuf.GeneratedMessageV3.

See an example on how the ProtoParquetReader is created line(65 and 69): 

and here how it is used (notice that is defined for only one record, the same one for multiple records would fail) 

 

**Reporter**: [Pau Alarcon ](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=paualarco)

**Note**: *This issue was originally created as [PARQUET-1871](https://issues.apache.org/jira/browse/PARQUET-1871). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.