apache / apache/paimon

NoSuchMethodError: org.apache.hadoop.hive.metastore.api.StorageDescriptor.setInputFormat[Bug]

Open
#3,216 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

0.7

### Compute Engine

flin 1.14.6

### Minimal reproduce step

CREATE CATALOG paimon_catalog WITH (
'type' = 'paimon',
'metastore' = 'hive',
'uri' = 'thrift://192.168.100.146:9083',
'metastore.partitioned-table' = 'true',
'location-in-properties' = 'true',
'table.type' = 'EXTERNAL'
);
USE CATALOG paimon_catalog;
CREATE TABLE my16 (
id BIGINT PRIMARY KEY NOT ENFORCED COMMENT 'ID' ,
name STRING COMMENT 'NAME' ,
zjhm STRING COMMENT 'ZJHM' ,
mz STRING COMMENT 'MZ' ,
age TINYINT COMMENT 'AGE' ,
sex TINYINT COMMENT 'SEX' ,
city STRING COMMENT 'CITY' ,
address STRING COMMENT 'ADDRESS' ,
jd DOUBLE COMMENT 'JD' ,
wd DOUBLE COMMENT 'WD' ,
lxdw STRING COMMENT 'LXDW' ,
costprice DOUBLE COMMENT 'COSTPRICE' ,
createtime TIMESTAMP COMMENT 'CREATETIME'
)
WITH (
'bucket' = '15',
'bucket-key' = 'id',
'changelog-producer' = 'input',
'num-sorted-run.compaction-trigger' = '10',
'write-buffer-size' = '1524mb',
'num-sorted-run.stop-trigger' = '2147483647',
'sort-spill-threshold' = '10',
'write-buffer-spillable' = 'true'
);

Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:201)
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.api.StorageDescriptor.setInputFormat(Ljava/lang/String;)V
at org.apache.paimon.hive.HiveCatalog.updateHmsTable(HiveCatalog.java:511)
at org.apache.paimon.hive.HiveCatalog.createTableImpl(HiveCatalog.java:383)
at org.apache.paimon.catalog.AbstractCatalog.createTable(AbstractCatalog.java:210)
at org.apache.paimon.flink.FlinkCatalog.createTable(FlinkCatalog.java:318)
at org.apache.flink.table.catalog.CatalogManager.lambda$createTable$10(CatalogManager.java:661)
at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:841)
at org.apache.flink.table.catalog.CatalogManager.createTable(CatalogManager.java:659)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:881)
at org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeOperation$3(LocalExecutor.java:209)
at org.apache.flink.table.client.gateway.context.ExecutionContext.wrapClassLoader(ExecutionContext.java:88)

![image](https://github.com/apache/paimon/assets/33175015/63213d93-ed8a-4c82-8090-d7d449dc1c2d)

### What doesn't meet your expectations?

can create table

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with HiveCatalog.updateHmsTable at HiveCatalog.java:511 and the createTableImpl path at line 383, using the Paimon 0.7 and Flink 1.14.6 reproduction. Investigate the Hive metastore dependency involved in StorageDescriptor.setInputFormat, then rerun the provided CREATE CATALOG and CREATE TABLE commands; done means table creation no longer fails with NoSuchMethodError.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.