apache / apache/parquet-java

ClassCastException

Open
#2,307 1 comment 0 reactions 0 assignees View on GitHub
Component: Java Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Hi

I'm writing protobuf to Parquet using ProtoParquetWriter, i'm also writing to local HDFS and write only  one file.

my code running in a bit data environment.

the problem is that after about 30 min of run i'm starting to get exception like this:

 

java.lang.ClassCastException: org.apache.parquet.io.MessageColumnIO cannot be cast to org.apache.parquet.io.PrimitiveColumnIO
 at org.apache.parquet.io.MessageColumnIO$MessageColumnIORecordConsumer.getColumnWriter(MessageColumnIO.java:432)
 at org.apache.parquet.io.MessageColumnIO$MessageColumnIORecordConsumer.addLong(MessageColumnIO.java:449)
 at org.apache.parquet.proto.ProtoWriteSupport$LongWriter.writeRawValue(ProtoWriteSupport.java:294)
 at org.apache.parquet.proto.ProtoWriteSupport$FieldWriter.writeField(ProtoWriteSupport.java:145)
 at org.apache.parquet.proto.ProtoWriteSupport$MessageWriter.writeAllFields(ProtoWriteSupport.java:229)
 at org.apache.parquet.proto.ProtoWriteSupport$MessageWriter.writeTopLevelMessage(ProtoWriteSupport.java:194)

 

I printed the message that cause to this error and I can't see any problem with it.

any idea?

 

**Reporter**: [the coder](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=theCoder)

**Note**: *This issue was originally created as [PARQUET-1541](https://issues.apache.org/jira/browse/PARQUET-1541). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.