apache / apache/parquet-java

Parquet-hive build fails on debian and OSX yosemite

Open
#1,771 5 comments 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

I am trying to compile the latest (1.6.0rc8 / git master -> PARQUET-214: Fix Avro string regression.) version of parquet-mr. The required versions of protobuf (2.5.0) and thrift (0.7) are installed and working properly (parquet-{protobuf,thrift} compile successfully). If I run mvn test, I get:

[ERROR] /Users/peltekov/Documents/incubator-parquet-mr/parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java:[46,20] cannot find symbol

[ERROR] symbol: class HiveBinding

[ERROR] location: package parquet.hive

[ERROR] /Users/peltekov/Documents/incubator-parquet-mr/parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java:[47,20] cannot find symbol

[ERROR] symbol: class HiveBindingFactory

[ERROR] location: package parquet.hive

[ERROR] /Users/peltekov/Documents/incubator-parquet-mr/parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java:[63,17] cannot find symbol

[ERROR] symbol: class HiveBinding

[ERROR] location: class org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper

[ERROR] /Users/peltekov/Documents/incubator-parquet-mr/parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java:[80,13] cannot find symbol

[ERROR] symbol: class HiveBinding

[ERROR] location: class org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper

Since I have no experience with hive, I am running out of ideas quite quickly. Obviously the generated files in parquet.hive are not included in the classpath when the tests are runned.

Also if I run
mvn clean install ; mvn test
then the tests are passed successfully.
And btw mvn compile fails in the same way as mvn test.

Is it a bug or I do something wrong?

**Reporter**: [Boris Peltekov](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=peltekster) / @peltekster

**Note**: *This issue was originally created as [PARQUET-231](https://issues.apache.org/jira/browse/PARQUET-231). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java and reproduce the failure with mvn compile or mvn test. Compare that with a preceding mvn clean install; done means the generated parquet.hive classes are available and compile and test succeed without requiring the clean install workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.