dbt-labs / dbt-labs/docs.getdbt.com
Improve dbt documentation with real-world examples for incremental models (Snowflake-focused)
- Dominant language
- JavaScript
- Stars
- 215
- Forks
- 1.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 130
Description
### Contributions
- [x] I have read the contribution docs, and understand what's expected of me.
### Link to the page on docs.getdbt.com requiring updates
Hi team,
I’d like to propose an improvement to the dbt documentation around incremental models.
### Problem
The current documentation explains incremental models well conceptually, but lacks real-world production examples, especially for Snowflake users.
In practice, data engineers often need to handle:
- Deduplication during incremental loads
- Late-arriving data
- Merge-based strategies with update conditions
- Idempotent pipeline design
These scenarios are common in production pipelines but are not clearly demonstrated in the current docs.
### Proposed improvement
Add a section under incremental models with practical examples such as:
- Deduplication using ROW_NUMBER()
- Merge logic with conditional updates
- Handling late-arriving records using timestamps
- Example Snowflake-based incremental model
### Example snippet (simplified)
```sql
{{ config(materialized='incremental', unique_key='id') }}
with source as (
select *
from {{ source('raw', 'orders') }}
),
deduplicated as (
select *
from (
select *,
row_number() over (partition by id order by updated_at desc) as rn
from source
)
where rn = 1
)
select *
from deduplicated
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}
### What part(s) of the page would you like to see updated?
The current incremental models documentation explains the concept and basic usage, but it lacks real-world production scenarios that data engineers commonly face.
Specifically, the following areas could be improved:
- There are no clear examples showing how to handle deduplication during incremental loads (e.g., using ROW_NUMBER or latest record selection).
- Late-arriving data handling is not demonstrated (e.g., records arriving after the initial load based on timestamps).
- Merge-based incremental strategies (especially for Snowflake) are not explained with practical SQL examples.
- There is limited guidance on building idempotent pipelines where reruns do not create duplicates or inconsistencies.
Why should the docs be changed:
In real-world data engineering pipelines, incremental models are rarely used in their simplest form. Engineers often need to combine filtering, deduplication, and merge logic to ensure data correctness. Without these examples, users may implement inefficient or incorrect patterns.
This improvement would support:
- Data engineers working with Snowflake and large-scale pipelines
- Teams implementing medallion architectures (RAW → staging → marts)
- Users trying to avoid duplication and maintain data consistency in incremental loads
Expected outcome:
- Add a new section or expand the existing incremental models page with practical, production-ready examples
- Include at least one Snowflake-based example demonstrating:
- Deduplication using window functions
- Incremental filtering using timestamps
- Merge logic with a unique key
- Improve clarity so users can directly apply these patterns in real-world pipelines
### Additional information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.