apache / apache/parquet-java

Proto-Parquet cannot write a schema with an empty group

Open
#2,142 1 comment 0 reactions 0 assignees View on GitHub
Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

This is valid protobuf:

 
```java

message Something {
    EmptyMessage outerEmptyMessage = 1;
}

message EmptyMessage {
}
```
However when we try to convert this to Parquet we get:
```java

org.apache.parquet.schema.InvalidSchemaException: Cannot write a schema with an empty group: optional group outerEmptyMessage = 1 {
}
```
Is this a limitation of the Parquet format, or just a bug?

If the former, can we break earlier when generating the schema out of Protobuf, and not when trying to write the content to the already generated schema?

 

**Reporter**: [André Pinto](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=andresp)

**Note**: *This issue was originally created as [PARQUET-1240](https://issues.apache.org/jira/browse/PARQUET-1240). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported protobuf example with an empty nested message and follow the protobuf-to-Parquet schema-generation path. Determine whether the generated empty group is unsupported by Parquet or produced incorrectly, and make the failure occur at schema generation if earlier validation is required.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.