dbt-labs / dbt-labs/docs.getdbt.com

Review docs for `incremental_strategy` for dbt-spark and dbt-databricks

Open
#4,639 0 comments 0 reactions 0 assignees View on GitHub
content improvement
Dominant language
JavaScript
Stars
215
Forks
1.2k
Avg merge
1d 15h
Merged PRs (30d)
130

Description

### Contributions

- [X] I have read the contribution docs, and understand what's expected of me.

### Link to the page on docs.getdbt.com requiring updates

Assess the following pages for accuracy and completeness in regards to incremental strategies:
- https://docs.getdbt.com/reference/resource-configs/databricks-configs#incremental-models
- https://docs.getdbt.com/reference/resource-configs/spark-configs#incremental-models

### What part(s) of the page would you like to see updated?

Assess if/how we should offer appropriate caveats based on endpoint or file format:
- dbt-spark supports `iceberg` as a file format; does dbt-databricks also?
- are all the relevant caveats included for which endpoints can be used with each incremental strategy?

### Additional information

- Caveats related to the incremental strategies usable with the Delta file format was added in [#77](https://github.com/dbt-labs/docs.getdbt.com/pull/77) (April 2020), but it has not updated since then to reflect additions of Hudi and Iceberg. Caveats related to file format were removed in [#4638](https://github.com/dbt-labs/docs.getdbt.com/pull/4638) in favor of this issue.
- dbt-spark and dbt-databricks allow the following combos of incremental strategy and file format (according to [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L19)):
- `merge` may have `delta` or `iceberg` or `hudi` for file format (see [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L46-L47))
- `insert_overwrite` is not allowed when connecting via endpoint (see [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L49-L50))

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.