apache / apache/parquet-java

[parquet-protobuf] Pass descriptor to ProtoWriteSupport constructor

Open
#2,507 0 comments 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: enhancement
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Currently, the ProtoWriteSupport class checks for Descriptor by calling `Protobufs.getMessageDescriptor` function which checks for descriptor in the classpath. There is no way to pass descriptor as an argument to the ProtoWriteSupport constructor.

In our approach to using parquet-mr library, we are using a descriptor that is not available in the classpath.
I will be happy to work on adding this support to the parquet-mr library.

**Reporter**: [Maulik Soneji](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mauliksoneji)

**Note**: *This issue was originally created as [PARQUET-1885](https://issues.apache.org/jira/browse/PARQUET-1885). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the ProtoWriteSupport constructor and its use of Protobufs.getMessageDescriptor. Trace how descriptors are currently discovered and identify the constructor behavior needed when a descriptor is supplied directly. Done means callers can provide a descriptor that is not available on the classpath.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.