ClickHouse / ClickHouse/dbt-clickhouse

Macro with `GRANT ... ON CLUSTER` fails when fetching results (dbt-clickhouse 1.10.2 regression)

Closed
#719 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

### Describe the bug
Since `dbt-clickhouse` 1.10.2, any query executed with `fetch=True` (e.g. via the `run_query()` dbt macro) that returns a distributed/cluster-DDL style HTTP response — such as `GRANT ... ON CLUSTER` — crashes with:

```
Runtime Error
ClickHouse exception: zip() argument 2 is longer than argument 1
```

This is a regression introduced by adding `strict=True` to a `zip()` call in `connections.py` without first validating that `column_names` and each result row are always the same length. For `ON CLUSTER` DDL, ClickHouse's HTTP interface returns a per-host status payload without matching column metadata, so the lengths don't line up, and the newly-strict `zip()` now raises instead of silently truncating.

### Root cause
`dbt/adapters/clickhouse/connections.py`, method `get_table_from_response()`:

```python
# dbt-clickhouse 1.10.1 and earlier (no crash):
data.append(dict(zip(column_names, row)))

# dbt-clickhouse 1.10.2 (crashes):
data.append(dict(zip(column_names, row, strict=True)))
```

This code path is only reached when a statement is executed with `fetch=True`, i.e. via `client.query()` rather than `client.command()`. In the dbt-clickhouse Jinja macros, this happens whenever a model/macro calls `{{ run_query(sql) }}` or `statement(..., fetch_result=True)`, as opposed to the default `statement()` call (which uses `fetch_result=False` → `.command()` → unaffected).

For a `GRANT ... ON CLUSTER ...` statement, the HTTP response is not a normal tabular `SELECT`-style result — it's a per-host execution status blob. Reproduced directly with clickhouse-connect:

```python
import clickhouse_connect
client = clickhouse_connect.get_client(host='', port=8443, username='', password='', secure=True)
res = client.query('grant on cluster bi_dwh dbt_developer to some_user', settings={'query_id': 'repro'})
print(res.column_names) # -> ()
print(res.result_set) # -> [['host1.example.com', '9440', '0', '', '1', '0\nhost2.example.com', '9440', '0', '', '0', '0']]
```

- column_names is an empty tuple ().
- result_set has one row with 11 raw fields (the two hosts' status fields, effectively concatenated with an embedded newline).

Zipping () (length 0) against an 11-element row with strict=True immediately raises `ValueError: zip() argument 2 is longer than argument 1`. With the old non-strict zip(), this silently produced an empty dict per row and no error — masking the mismatch but not crashing.

### Steps to reproduce
1. Configure a profiles.yml target against a ClickHouse cluster with cluster: set (so ON CLUSTER clauses are emitted) and dbt-clickhouse==1.10.2 installed.
2. Run any dbt macro that executes cluster DDL via a fetch-enabled call path, for example:

```sql
{% macro repro_bug(user) %}
{% set grant_sql %}
grant on cluster {{ target.cluster }} some_role to {{ user }};
{% endset %}
{% do run_query(grant_sql) %}
{% endmacro %}
```

```shell
dbt run-operation repro_bug --args 'user: some_user'
```

3. Observe:
```
Encountered an error while running operation: Runtime Error
ClickHouse exception: zip() argument 2 is longer than argument 1
Full traceback bottoms out at:
File ".../dbt/adapters/clickhouse/connections.py", line 80, in get_table_from_response
data.append(dict(zip(column_names, row, strict=True)))
ValueError: zip() argument 2 is longer than argument 1
```

4. Confirm the DDL itself actually succeeded server-side despite the client-side crash, e.g.:
```
SELECT * FROM system.role_grants WHERE user_name = 'some_user';
```
— the grant is present, showing this is a purely client-side response-parsing bug, not a failed/rolled-back statement.

## Minimal reproduction without dbt

```python
import clickhouse_connect

client = clickhouse_connect.get_client(
host='', port=8443, username='', password='', secure=True
)
res = client.query("grant on cluster to ")
print(res.column_names, res.result_set)

# Now reproduce dbt-clickhouse's exact failing line:
dict(zip(res.column_names, res.result_set[0], strict=True))
# -> ValueError: zip() argument 2 is longer than argument 1
```

### Expected behaviour
Macros using `GRANT ... ON CLUSTER ...` to work

### Code examples, such as models or profile settings
Please see "Steps to reproduce".

### dbt and/or ClickHouse server logs

### Configuration
#### Environment

- `dbt-clickhouse`: **1.10.2** (bug present) / 1.10.1 (bug absent — confirmed working)
- `dbt-core`: 1.11.12
- `dbt-adapters`: 1.22.10
- `clickhouse-connect`: reproduced on both 0.10.0 and 1.7.2 (not a clickhouse-connect version issue)
- Python: 3.12.13
- OS: macOS (Darwin 25.6.0, arm64)

#### ClickHouse server
- ClickHouse server: 26.7.3.19, official build
- Cluster: 1 shard, 2 replicas (`ON CLUSTER` DDL enabled)

Contributor guide

Open the contributing guide

Research direction

Start in dbt/adapters/clickhouse/connections.py at get_table_from_response(), then reproduce the issue with the minimal clickhouse-connect example or the run_query macro described in the report. Compare the response's empty column_names with its populated result_set and verify that a fetch-enabled GRANT ... ON CLUSTER completes without raising while normal fetched rows remain handled.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, python
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.