apache / apache/hudi

[SUPPORT]Flink HiveCatalog should respect 'managed_table' options to avoid deleting data unexpectable.

Open
#7,936 1 comment 0 reactions 1 assignee Claimed by @danny0405 View on GitHub
component:catalog-sync engine:flink engine:hive priority:high
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**

Currently it is unacceptable when using drop table statement which the table managed by hive catalog will unexpectedly deleting data.

There is also an options named "hoodie.datasource.hive_sync.create_managed_table" but not been honored by hive catalog.
The catalog currently always deleting the data, which should be changed to honored by the "managed_table" option.

```JAVA HoodieHiveCatalog.java
@Override
public void dropTable(ObjectPath tablePath, boolean ignoreIfNotExists)
throws TableNotExistException, CatalogException {
checkNotNull(tablePath, "Table path cannot be null");

try {
client.dropTable(
tablePath.getDatabaseName(),
tablePath.getObjectName(),
// Indicate whether associated data should be deleted.
// Set to 'true' for now because Flink tables shouldn't have data in Hive. Can
// be changed later if necessary
true,
ignoreIfNotExists);
} catch (NoSuchObjectException e) {
if (!ignoreIfNotExists) {
throw new TableNotExistException(getName(), tablePath);
}
} catch (TException e) {
throw new HoodieCatalogException(
String.format("Failed to drop table %s", tablePath.getFullName()), e);
}
}
```

**To Reproduce**

Steps to reproduce the behavior:

1. create hive catalog in flink.
2. drop the table in hive catalog by "DROP TABLE hive_catalog.testing_table;"

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment Description**

* Hudi version : 0.13.0

* Spark version : N/A

* Hive version : 3.1.3

* Hadoop version : 3.1.2

* Storage (HDFS/S3/GCS..) : OSS

* Running on Docker? (yes/no) : no

**Additional context**

**Stacktrace**

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.