apache / apache/parquet-java

parquet-cli not working in release 1.12.0

Open
#2,630 2 comments 0 reactions 0 assignees View on GitHub
Component: CLI Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

When I run parquet-cli getting  java.lang.NoSuchMethodError

Steps to repdouce:
1. Download parquet-mr 1.12.0 from
1. Build and install using mvn clean install
1. cd parquet-cli
1. `mvn dependency:copy-dependencies`
1. `java -cp 'target/*:target/dependency/*' org.apache.parquet.cli.Main head `
1. Got below exception

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil ([file:/home/amsundar/hgroot/parquet-mr-apache-parquet-1.12.0/parquet-cli/target/dependency/hadoop-auth-2.10.1.jar\|file://home/amsundar/hgroot/parquet-mr-apache-parquet-1.12.0/parquet-cli/target/dependency/hadoop-auth-2.10.1.jar]) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.parquet.avro.AvroSchemaConverter.convert(Lorg/apache/parquet/schema/MessageType;)Lorg/apache/avro/Schema;
at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:89)
at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:405)
at org.apache.parquet.cli.commands.CatCommand.run(CatCommand.java:66)
at org.apache.parquet.cli.Main.run(Main.java:155)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.parquet.cli.Main.main(Main.java:185)

 

 

**Reporter**: [Akshay Sundarraj](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=akshaysu)
#### Original Issue Attachments:
- [sample.parquet](https://issues.apache.org/jira/secure/attachment/13030721/sample.parquet)
- [sample.schema](https://issues.apache.org/jira/secure/attachment/13030722/sample.schema)

**Note**: *This issue was originally created as [PARQUET-2065](https://issues.apache.org/jira/browse/PARQUET-2065). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure with the listed Maven commands and sample.parquet, then inspect parquet-cli/util/Schemas.java at the fromParquet call and the AvroSchemaConverter API involved in the NoSuchMethodError. Done means the parquet-cli command completes successfully against the provided sample file without this exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.