apache / apache/parquet-java

ParquetMetadataConverter should never log at level WARN

Open
#3,025 0 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

### Describe the bug, including details regarding any error messages, version, and platform.

When ParquetMetadataConverter encounters a mismatch between logical and converted type, it logs a message at level WARN. https://github.com/apache/parquet-java/blob/eed26b4ea96881cc543b4233cdddece5d1d608d7/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java#L1864

This behavior does not make sense because one of two things must be true:

1. This is an issue that can affect normal operation and the application needs to (potentially) respond to it
2. This is not a concern for the calling application and it won't impact normal operation

If item 1 is true, then an exception should be thrown and the application choose to respond or not (possibly by logging). If item item 2 is true, then the log message should either be omitted entirely or be logged at level DEBUG (INFO level would still be too spammy)

I would be happy to submit a patch if there is consensus about which approach to take. Perhaps there should be a flag to control whether the exception should be thrown to avoid breaking existing code.

### Component(s)

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java at the referenced line and inspect the surrounding metadata conversion behavior. Check relevant converter tests and logging conventions, then confirm with maintainers whether the warning should be removed, downgraded, or replaced with an exception. Done means the agreed behavior is implemented without unintended compatibility changes and covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.