dbt-labs / dbt-labs/dbt-adapters

[Feature] Introduce ha flag for table materialisation and iceberg tables to allow users to control final location

Open
#433 0 comments 2 reactions 0 assignees View on GitHub
feature:iceberg pkg:dbt-athena
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this your first time submitting a feature request?

- [X] I have searched the existing issues, and I could not find an existing issue for this feature

### Describe the feature

The current behaviour for table_type=iceberg and materialization=table implements high availability by default, this means that a new table is created with a not final name in a unique location, then the final target table is renamed. As consequence of this implementation we need to force the user of using unique locations.

> **What if the user don't want to have high availability and specific locations to be fully in control of where the data is stored?**

Adding the possibility to use the ha flag for iceberg table allow the user to be more in control.

Here the proposal - note only for iceberg tables, as they are the affected one:
* ha=true -> follow the current implementation
* ha=false -> drop the table and create the table to the final location - doing so we allow the user to use external_location, or data_naming: table, schema_table or whatever they want.

### Describe alternatives you've considered

_No response_

### Who will this benefit?

Users that want to control final iceberg table location in S3.

### Are you interested in contributing this feature?

_No response_

### Anything else?

The current behaviour for iceberg tables is set to "ha=true", maintain this behavior to avoid breaking changes for the users.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.