ClickHouse / ClickHouse/dbt-clickhouse

Allow for Custom Incremental Strategies

Open
#545 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

**Is your feature request related to a problem? Please describe.**

I have some dictionaries based on ClickHouse tables which are also part of the dbt DAG (materialization = table). These tables are small enough to be recreated upon every run. However, these runs frequently fail because the dictionary's underlying table may not be removed/renamed/altered. Thus, I wanted to make use of a [custom incremental strategy](https://docs.getdbt.com/docs/build/incremental-strategy#custom-strategies) that would simply truncate and insert all data from the staging view. Unfortunately this is impossible, as ClickHouse dbt adapter says "The incremental strategy 'truncate_insert' is not valid for ClickHouse."

**Describe the solution you'd like**

Being able to use custom incremental strategy as documented in dbt manual.

**Describe alternatives you've considered**

- Not using dictionaries based on dbt models
- Using insert and delete incremental strategy

Contributor guide

Open the contributing guide

Research direction

Start by tracing where the ClickHouse dbt adapter validates incremental strategies, using the reported "truncate_insert" rejection and the linked dbt custom-strategies documentation as context. Done means custom incremental strategies are accepted and can support the requested truncate-and-insert workflow without rejecting the strategy as invalid.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, python
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.