ClickHouse: implement source freshness (ClickHouseMetadataAdapter::freshness_inner)
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
## Context
`dbt source freshness` panics for ClickHouse with:
```
not yet implemented: ClickHouseAdapter::freshness
--> fs/sa/crates/dbt-adapter/src/metadata/clickhouse/mod.rs:263
```
The `freshness_inner` method on `ClickHouseMetadataAdapter` is a `todo!()`:
```rust
fn freshness_inner(
&self,
_relations: &[Arc],
_token: CancellationToken,
) -> AsyncAdapterResult<'_, BTreeMap> {
todo!("ClickHouseAdapter::freshness")
}
```
This means any project that configures source freshness on a ClickHouse source will hard-crash the engine.
## What's needed
Implement `freshness_inner` for ClickHouse — query each relation's last-modified timestamp and return a `MetadataFreshness` per relation. ClickHouse exposes this via `system.parts` or `system.tables` (the `metadata_modification_time` column).
## Location
`fs/sa/crates/dbt-adapter/src/metadata/clickhouse/mod.rs` — `freshness_inner` method.
## Tracking
The `adapters_clickhouse_metadata_source_freshness` test exists in `crates/dbt-adapter-tests/tests/adapter/clickhouse.rs` and is `#[ignore]`'d with this issue as the blocker. Un-ignore and record once implemented.
Contributor guide
Assessment
This issue has not been assessed yet.