ClickHouse / ClickHouse/dbt-clickhouse

Running DBT on MacOS with dbt-clickhouse does not support renameat2()

Open
#340 7 comments 2 reactions 0 assignees View on GitHub
bug good first issue
Dominant language
Python
Stars
362
Forks
177
Avg merge
2d 10h
Merged PRs (30d)
8

Description

### Describe the bug
When running : `dbt run --target=clickhouse-dev --select=dbt_mart` on MacOS Sonoma 14.6.1 (MacBook Pro (13-inch, M1, 2020))

```
08:27:45 Running with dbt=1.7.17
08:27:46 Registered adapter: clickhouse=1.7.7
08:27:46 Found 65 models, 3 seeds, 280 tests, 85 sources, 0 exposures, 0 metrics, 942 macros, 0 groups, 0 semantic models
08:27:46
08:27:48 dbt_clickhouse adapter: ClickHouse server does not support the EXCHANGE TABLES command
08:27:48 dbt_clickhouse adapter: This can be caused by an obsolete ClickHouse version or by running ClickHouse on
08:27:48 dbt_clickhouse adapter: an operating system that does not support the low level renameat2() system call.
08:27:48 dbt_clickhouse adapter: Some DBT materializations will be slower and not atomic as a result.
08:27:49 dbt_clickhouse adapter: Unexpected server exception dropping table
08:27:49
08:27:49 Finished running in 0 hours 0 minutes and 2.68 seconds (2.68s).
08:27:49 Encountered an error:
Database Error
'NoneType' object has no attribute 'version_major'
make: *** [dbt-run] Error 2
```

With :
- clickhouse : 23.8.4.69
- dbt-clickhouse : 1.7.7
- dbt-core : 1.7.17

While the same code is running perfectly in Linux distributions.

### Steps to reproduce
1. Get a MacOS Sonoma 14.6.1
2. Create a DBT project
3. Create a clickhouse-server instance with version 23.8.4.69
4. Create a connection profile for this instance called "my_profile"
5. Create a dummy model called "my_mart" in the DBT project
6. Execute `dbt run --target=my_profile --select=my_mart`

### Expected behaviour
A correct DBT materialization

### Code examples, such as models or profile settings
profiles.yml
```
clickhouse-dev:
type: clickhouse
driver: native
secure: true
verify: false
schema: dbt
retries: 3
connect_timeout: 100
send_receive_timeout: 3000
sync_request_timeout: 15
threads: 4
host: localhost # SSH tunneling
port: 9430 # ssh port mapping
user: user
password: password
custom_settings: {max_bytes_before_external_group_by: '68719480000', max_bytes_before_external_sort: '68719480000'}
```

### dbt and/or ClickHouse server logs
```
08:27:45 Running with dbt=1.7.17
08:27:46 Registered adapter: clickhouse=1.7.7
08:27:46 Found 65 models, 3 seeds, 280 tests, 85 sources, 0 exposures, 0 metrics, 942 macros, 0 groups, 0 semantic models
08:27:46
08:27:48 dbt_clickhouse adapter: ClickHouse server does not support the EXCHANGE TABLES command
08:27:48 dbt_clickhouse adapter: This can be caused by an obsolete ClickHouse version or by running ClickHouse on
08:27:48 dbt_clickhouse adapter: an operating system that does not support the low level renameat2() system call.
08:27:48 dbt_clickhouse adapter: Some DBT materializations will be slower and not atomic as a result.
08:27:49 dbt_clickhouse adapter: Unexpected server exception dropping table
08:27:49
08:27:49 Finished running in 0 hours 0 minutes and 2.68 seconds (2.68s).
08:27:49 Encountered an error:
Database Error
'NoneType' object has no attribute 'version_major'
make: *** [dbt-run] Error 2
```

### Configuration
#### Environment
* dbt version: 1.7.17
* dbt-clickhouse version: 1.7.7
* clickhouse-driver version (if using native): 0.2.7
* clickhouse-connect version (if using http): 0.7.11
* Python version: 3.11
* Operating system: Sonoma 14.6.1

#### ClickHouse server
* ClickHouse Server version: 23.8.4.69

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported `dbt run --target=my_profile --select=my_mart` failure on macOS with ClickHouse 23.8.4.69, then trace the EXCHANGE TABLES fallback and the `'NoneType' object has no attribute 'version_major'` error. Done means the dummy model receives a correct DBT materialization on macOS without the reported exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.