apache / apache/hudi

[SUPPORT]PrestoDB failed to query data from mor table

Open
#8,078 3 comments 0 reactions 1 assignee Claimed by @codope View on GitHub
area:query-engine component:trino priority:high type:feature
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 4h
Merged PRs (30d)
112

Description

An error occurred when PrestoDB queried a field with TIMESTAMP type in the MOR table [Snapshot Mode]

```
class org.apache.hadoop.io.LongWritable cannot be cast to class org.apache.hadoop.hive.serde2.io.TimestampWritable (org.apache.hadoop.io.LongWritable and org.apache.hadoop.hive.serde2.io.TimestampWritable are in unnamed module of loader com.facebook.presto.server.PluginClassLoader @648ec9f4)
```

**To Reproduce**

Steps to reproduce the behavior:

1.
```
drop table if exists `default`.`spark_0_12_1_test2`;
```
2.
```
CREATE TABLE `default`.`spark_0_12_1_test2` (
`id` INT
,`name` string
,`age` INT
,`sync_time` TIMESTAMP
) USING HUDI
TBLPROPERTIES (
`type` = 'mor'
,`primaryKey` = 'id'
,`preCombineField` = 'sync_time'
,`hoodie.bucket.index.hash.field` = 'id'
,`hoodie.index.type` = 'BLOOM'
,`hoodie.datasource.write.hive_style_partitioning` = 'false'
,`hoodie.compaction.payload.class` = 'org.apache.hudi.common.model.OverwriteWithLatestAvroPayload',
`hoodie.datasource.hive_sync.support_timestamp` = 'true'
)
COMMENT 'test_0.12.1';
```
3.
```
INSERT INTO `default`.`spark_0_12_1_test2`(id, name, age, sync_time) SELECT 2 AS id, 'hudi0121' AS name, 22 AS age, NOW() AS sync_time;

update `default`.`spark_0_12_1_test2` set age = 33, sync_time = now() where id = 2;
```
4. query with prestodb
```
select * from hudi."default".spark_0_12_1_test2;
```

**Environment Description**

* Hudi version : 0.12.1

* Spark version : 3.1.3

* PrestoDB version: 0.279

* Hive version : 3.1.0

* Hadoop version : 3.1.1

* Storage (HDFS/S3/GCS..) : HDFS

* Running on Docker? (yes/no) : no

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.