ClickHouse / ClickHouse/dbt-clickhouse

ttl not added to local table in distributed_incremental materialization

Open
#321 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

### Describe the bug

Tried to materialize table with `distributed_incremental` materialization.
`TTL` setting doesn't exist for local table.

### Steps to reproduce
1. Add model `test_model` with config. Run `dbt build`
```sql
{{ config(
materialized='distributed_incremental',
engine='ReplicatedReplacingMergeTree(datetime)',
incremental_strategy='append',
order_by='datetime, object_id',
partition_by='toYYYYMM(datetime)',
ttl='toStartOfYear(datetime + toIntervalYear(5))',
sharding_key='cityHash64(object_id)'
) }}

SELECT * FROM {{ source('default', 'stg_test_table') }}
```
2. Check local table with
```sql
SHOW CREATE default.test_model_local
```
3. Engine will be
```sql
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
SETTINGS index_granularity = 8192
```

### Expected behaviour

Engine of local table should contain TTL setting
```sql
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
TTL toStartOfYear(datetime + toIntervalYear(5))
SETTINGS index_granularity = 8192
```

### Code examples, such as models or profile settings

```yaml
version: 2

sources:
- name: default
tables:
- name: stg_test_table
```

### dbt and/or ClickHouse server logs

### Configuration
#### Environment
* dbt version: 1.7.15
* dbt-clickhouse version: 1.7.7
* clickhouse-driver version (if using native): 0.2.6
* clickhouse-connect version (if using http): 0.7.12
* Python version: 3.10
* Operating system: python:3.10-slim docker image

#### ClickHouse server
* ClickHouse Server version: 24.3.4.147
* `CREATE TABLE` statements for tables involved:

```sql
CREATE TABLE default.stg_test_table ON CLUSTER '{cluster}'
(
object_id String,
datetime DateTime,
field1 String,
field2 String
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}')
ORDER BY object_id;
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.