ClickHouse / ClickHouse/dbt-clickhouse
Allow for Custom Incremental Strategies
- Dominant language
- Python
- Stars
- 362
- Forks
- 177
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 8
Description
**Is your feature request related to a problem? Please describe.**
I have some dictionaries based on ClickHouse tables which are also part of the dbt DAG (materialization = table). These tables are small enough to be recreated upon every run. However, these runs frequently fail because the dictionary's underlying table may not be removed/renamed/altered. Thus, I wanted to make use of a [custom incremental strategy](https://docs.getdbt.com/docs/build/incremental-strategy#custom-strategies) that would simply truncate and insert all data from the staging view. Unfortunately this is impossible, as ClickHouse dbt adapter says "The incremental strategy 'truncate_insert' is not valid for ClickHouse."
**Describe the solution you'd like**
Being able to use custom incremental strategy as documented in dbt manual.
**Describe alternatives you've considered**
- Not using dictionaries based on dbt models
- Using insert and delete incremental strategy
Contributor guide
Research direction
Start by tracing where the ClickHouse dbt adapter validates incremental strategies, using the reported "truncate_insert" rejection and the linked dbt custom-strategies documentation as context. Done means custom incremental strategies are accepted and can support the requested truncate-and-insert workflow without rejecting the strategy as invalid.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100