dbt-labs / dbt-labs/docs.getdbt.com
Review docs for `incremental_strategy` for dbt-spark and dbt-databricks
- Dominant language
- JavaScript
- Stars
- 215
- Forks
- 1.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 130
Description
### Contributions
- [X] I have read the contribution docs, and understand what's expected of me.
### Link to the page on docs.getdbt.com requiring updates
Assess the following pages for accuracy and completeness in regards to incremental strategies:
- https://docs.getdbt.com/reference/resource-configs/databricks-configs#incremental-models
- https://docs.getdbt.com/reference/resource-configs/spark-configs#incremental-models
### What part(s) of the page would you like to see updated?
Assess if/how we should offer appropriate caveats based on endpoint or file format:
- dbt-spark supports `iceberg` as a file format; does dbt-databricks also?
- are all the relevant caveats included for which endpoints can be used with each incremental strategy?
### Additional information
- Caveats related to the incremental strategies usable with the Delta file format was added in [#77](https://github.com/dbt-labs/docs.getdbt.com/pull/77) (April 2020), but it has not updated since then to reflect additions of Hudi and Iceberg. Caveats related to file format were removed in [#4638](https://github.com/dbt-labs/docs.getdbt.com/pull/4638) in favor of this issue.
- dbt-spark and dbt-databricks allow the following combos of incremental strategy and file format (according to [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L19)):
- `merge` may have `delta` or `iceberg` or `hudi` for file format (see [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L46-L47))
- `insert_overwrite` is not allowed when connecting via endpoint (see [here](https://github.com/dbt-labs/dbt-spark/blob/ffc24ba30aa87e38e42ced897ccb19961425ad81/dbt/include/spark/macros/materializations/incremental/validate.sql#L49-L50))
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.