ClickHouse / ClickHouse/dbt-clickhouse

Support dbt Fusion

Open
#555 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

dbt Fusion is still in its early days. Most of their content is still in beta and they still don't have an official way to implement the new version of the adapter for ClickHouse. As far as we know, we need at least these things:
- ADBC API: ClickHouse currently have it available but it's still in its early days and not ready for dbt workloads. Work is being done to improve it. dbt folks also are supposed to open source `XDBC` which will be the alternative so we can use ODBC until ADBC is ready. It's still not open sourced.
- ANTLR Grammars: dbt still need to tell us what exactly they need. We may be able to use this ANTLR already present in the CH repository here, but we need to validate if they can be fully used for this https://github.com/ClickHouse/ClickHouse/tree/master/utils/antlr
- Implement new Macros: We still need information about which macros we need to implement. For example we need to probably implement changes for `dbt-frontend-common` and `dbt-sql` as they are related to the SQL dialect features.
- Guides for the rest of operations: for example, we need to know the process to send the new adapter to dbt Labs so it can be signed by them https://github.com/dbt-labs/dbt-core/issues/13138.

We have not releases dates for any of these topics yet, but we hope we could start working on them soon.

Sources:
- https://docs.getdbt.com/blog/dbt-fusion-engine-components

Contributor guide

Open the contributing guide

Research direction

Start by reading the dbt Fusion components source and the ClickHouse ANTLR grammars under utils/antlr. Clarify with dbt which grammars, macros, and adapter APIs are required, then determine the signing and submission process described in dbt-core issue 13138. Done requires a defined implementation scope and confirmed integration path; the issue currently provides neither.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, python, sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.