dbt-labs / dbt-labs/docs.getdbt.com

Document unpartitioned Hive `insert_overwrite` behavior for Athena

Open
#9,643 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
215
Forks
1.2k
Avg merge
1d 15h
Merged PRs (30d)
130

Description

### Contributions

- [x] I have read the contribution docs, and understand what's expected of me.

### Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/reference/resource-configs/athena-configs

### What part(s) of the page would you like to see updated?

The "Incremental table models" section currently documents `insert_overwrite` as always falling back to `append` when no partitions are defined.

dbt-labs/dbt-adapters#2082 proposes distinguishing an omitted strategy from an explicitly configured `insert_overwrite` for unpartitioned Hive models:

- When the strategy is omitted, preserve the existing `append` fallback.
- When `insert_overwrite` is explicit and dbt creates a new S3 location per run, which is the default, replace the complete table.
- When the model reuses an S3 path through `external_location` or `s3_data_naming`, preserve `append` and emit an informational message.
- Preserve existing behavior for partitioned Hive models and all other strategies.

The Athena configuration page should document this distinction when the adapter change is released.

### Suggested replacement copy

Replace the current `insert_overwrite` bullet with:

- `insert_overwrite` (default for Hive): For partitioned models, dbt deletes overlapping partitions from the destination table and inserts the new records. For unpartitioned models, an explicitly configured `incremental_strategy='insert_overwrite'` replaces the complete table when dbt creates a new S3 location per run (`s3_data_naming` contains `unique`, which is the default, and `external_location` is not configured). If the strategy is omitted, dbt preserves the historical `append` fallback. Models that explicitly request `insert_overwrite` but reuse the same S3 path also preserve `append` and emit an informational message.

### Additional information

Feature issue: https://github.com/dbt-labs/dbt-adapters/issues/2081

Implementation PR: https://github.com/dbt-labs/dbt-adapters/pull/2082

The dbt-athena README is also updated in the implementation PR.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.