apache / apache/parquet-java

GroupType.union(Type toMerge, boolean strict) does not honor strict parameter

Open
#1,901 7 comments 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

This is the code as it currently stands in master:

```Java
@Override
protected Type union(Type toMerge, boolean strict) {
if (toMerge.isPrimitive()) {
throw new IncompatibleSchemaModificationException("can not merge primitive type " + toMerge + " into group type " + this);
}
return new GroupType(toMerge.getRepetition(), getName(), mergeFields(toMerge.asGroupType()));
}
```

Note the call to `mergeFields` omits the `strict` parameter. I believe the code should be:

```Java
@Override
protected Type union(Type toMerge, boolean strict) {
if (toMerge.isPrimitive()) {
throw new IncompatibleSchemaModificationException("can not merge primitive type " + toMerge + " into group type " + this);
}
return new GroupType(toMerge.getRepetition(), getName(), mergeFields(toMerge.asGroupType(), strict));
}
```

Note the call to `mergeFields` includes the `strict` parameter.

I would work on this myself, but I'm having considerable trouble working with the codebase (see e.g. http://stackoverflow.com/questions/31229445/build-failure-apache-parquet-mr-source-mvn-install-failure). Given the (assumed) simplicity of the fix, can a seasoned Parquet contributor take this up? Cheers.

**Reporter**: [Michael MacFadden](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=michael) / @mallman

**Note**: *This issue was originally created as [PARQUET-390](https://issues.apache.org/jira/browse/PARQUET-390). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Search the Parquet Java codebase for GroupType.union(Type toMerge, boolean strict), then inspect how mergeFields is called from that method. Confirm the strict value is propagated as described and run the relevant project tests to verify strict and non-strict schema merging behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.