ClickHouse / ClickHouse/dbt-clickhouse
Macro with `GRANT ... ON CLUSTER` fails when fetching results (dbt-clickhouse 1.10.2 regression)
- Dominant language
- Python
- Stars
- 362
- Forks
- 177
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 8
Description
### Describe the bug
Since `dbt-clickhouse` 1.10.2, any query executed with `fetch=True` (e.g. via the `run_query()` dbt macro) that returns a distributed/cluster-DDL style HTTP response — such as `GRANT ... ON CLUSTER` — crashes with:
```
Runtime Error
ClickHouse exception: zip() argument 2 is longer than argument 1
```
This is a regression introduced by adding `strict=True` to a `zip()` call in `connections.py` without first validating that `column_names` and each result row are always the same length. For `ON CLUSTER` DDL, ClickHouse's HTTP interface returns a per-host status payload without matching column metadata, so the lengths don't line up, and the newly-strict `zip()` now raises instead of silently truncating.
### Root cause
`dbt/adapters/clickhouse/connections.py`, method `get_table_from_response()`:
```python
# dbt-clickhouse 1.10.1 and earlier (no crash):
data.append(dict(zip(column_names, row)))
# dbt-clickhouse 1.10.2 (crashes):
data.append(dict(zip(column_names, row, strict=True)))
```
This code path is only reached when a statement is executed with `fetch=True`, i.e. via `client.query()` rather than `client.command()`. In the dbt-clickhouse Jinja macros, this happens whenever a model/macro calls `{{ run_query(sql) }}` or `statement(..., fetch_result=True)`, as opposed to the default `statement()` call (which uses `fetch_result=False` → `.command()` → unaffected).
For a `GRANT ... ON CLUSTER ...` statement, the HTTP response is not a normal tabular `SELECT`-style result — it's a per-host execution status blob. Reproduced directly with clickhouse-connect:
```python
import clickhouse_connect
client = clickhouse_connect.get_client(host='', port=8443, username='', password='', secure=True)
res = client.query('grant on cluster bi_dwh dbt_developer to some_user', settings={'query_id': 'repro'})
print(res.column_names) # -> ()
print(res.result_set) # -> [['host1.example.com', '9440', '0', '', '1', '0\nhost2.example.com', '9440', '0', '', '0', '0']]
```
- column_names is an empty tuple ().
- result_set has one row with 11 raw fields (the two hosts' status fields, effectively concatenated with an embedded newline).
Zipping () (length 0) against an 11-element row with strict=True immediately raises `ValueError: zip() argument 2 is longer than argument 1`. With the old non-strict zip(), this silently produced an empty dict per row and no error — masking the mismatch but not crashing.
### Steps to reproduce
1. Configure a profiles.yml target against a ClickHouse cluster with cluster: set (so ON CLUSTER clauses are emitted) and dbt-clickhouse==1.10.2 installed.
2. Run any dbt macro that executes cluster DDL via a fetch-enabled call path, for example:
```sql
{% macro repro_bug(user) %}
{% set grant_sql %}
grant on cluster {{ target.cluster }} some_role to {{ user }};
{% endset %}
{% do run_query(grant_sql) %}
{% endmacro %}
```
```shell
dbt run-operation repro_bug --args 'user: some_user'
```
3. Observe:
```
Encountered an error while running operation: Runtime Error
ClickHouse exception: zip() argument 2 is longer than argument 1
Full traceback bottoms out at:
File ".../dbt/adapters/clickhouse/connections.py", line 80, in get_table_from_response
data.append(dict(zip(column_names, row, strict=True)))
ValueError: zip() argument 2 is longer than argument 1
```
4. Confirm the DDL itself actually succeeded server-side despite the client-side crash, e.g.:
```
SELECT * FROM system.role_grants WHERE user_name = 'some_user';
```
— the grant is present, showing this is a purely client-side response-parsing bug, not a failed/rolled-back statement.
## Minimal reproduction without dbt
```python
import clickhouse_connect
client = clickhouse_connect.get_client(
host='', port=8443, username='', password='', secure=True
)
res = client.query("grant on cluster to ")
print(res.column_names, res.result_set)
# Now reproduce dbt-clickhouse's exact failing line:
dict(zip(res.column_names, res.result_set[0], strict=True))
# -> ValueError: zip() argument 2 is longer than argument 1
```
### Expected behaviour
Macros using `GRANT ... ON CLUSTER ...` to work
### Code examples, such as models or profile settings
Please see "Steps to reproduce".
### dbt and/or ClickHouse server logs
### Configuration
#### Environment
- `dbt-clickhouse`: **1.10.2** (bug present) / 1.10.1 (bug absent — confirmed working)
- `dbt-core`: 1.11.12
- `dbt-adapters`: 1.22.10
- `clickhouse-connect`: reproduced on both 0.10.0 and 1.7.2 (not a clickhouse-connect version issue)
- Python: 3.12.13
- OS: macOS (Darwin 25.6.0, arm64)
#### ClickHouse server
- ClickHouse server: 26.7.3.19, official build
- Cluster: 1 shard, 2 replicas (`ON CLUSTER` DDL enabled)
Contributor guide
Research direction
Start in dbt/adapters/clickhouse/connections.py at get_table_from_response(), then reproduce the issue with the minimal clickhouse-connect example or the run_query macro described in the report. Compare the response's empty column_names with its populated result_set and verify that a fetch-enabled GRANT ... ON CLUSTER completes without raising while normal fetched rows remain handled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, python
- Domain
- database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100