apache / apache/parquet-java

Not able to do backwards compatible projection

Open
#2,255 0 comments 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Currently the following scenarios fail while using projections. This makes it hard to do backwards compatible schema changes like adding an optional field or making a required field optional.

1. Adding an optional field to an existing thrift schema and making it part of the projection for reading data written using older thrift.

2. Projecting a required field using optional.

**Reporter**: [Mouli Mukherjee](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=moulim)
#### PRs and other links:
- [GitHub Pull Request #558](https://github.com/apache/parquet-mr/pull/558)

**Note**: *This issue was originally created as [PARQUET-1464](https://issues.apache.org/jira/browse/PARQUET-1464). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.