apache / apache/iceberg

Should behaviour of `dropTable` be up to REST Catalog implementation?

Open
#16,564 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Query engine

Trino version 481

### Question

Hi,
recently I have been playing around with several implementations of REST Catalog and I noticed that not everybody follows the spec when it comes to `purgeRequested` flag in `RESTSessionCatalog#dropTable(SessionContext, TableIdentifier)`. This specifically affects how `unregister_table` procedure behaves in Trino which calls that `dropTable` method.

For example Lakekeeper sets default value of `purgeRequested` to `true` meaning calling `unregisted_table` ,which by documentation and definition should only affect metadata and keep the data untouched, results in deleting data and can cause accidental data loss if someone migrated from one REST catalog implementation to another and expected the procedure the behave the same.

As I'm fairly new to iceberg I would like to hear what the consensus is on that behaviour. Should the difference between purgeTable and dropTable be up to catalogs implementation?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.