ClickHouse / ClickHouse/dbt-clickhouse
Running DBT on MacOS with dbt-clickhouse does not support renameat2()
- Dominant language
- Python
- Stars
- 362
- Forks
- 177
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 8
Description
### Describe the bug
When running : `dbt run --target=clickhouse-dev --select=dbt_mart` on MacOS Sonoma 14.6.1 (MacBook Pro (13-inch, M1, 2020))
```
08:27:45 Running with dbt=1.7.17
08:27:46 Registered adapter: clickhouse=1.7.7
08:27:46 Found 65 models, 3 seeds, 280 tests, 85 sources, 0 exposures, 0 metrics, 942 macros, 0 groups, 0 semantic models
08:27:46
08:27:48 dbt_clickhouse adapter: ClickHouse server does not support the EXCHANGE TABLES command
08:27:48 dbt_clickhouse adapter: This can be caused by an obsolete ClickHouse version or by running ClickHouse on
08:27:48 dbt_clickhouse adapter: an operating system that does not support the low level renameat2() system call.
08:27:48 dbt_clickhouse adapter: Some DBT materializations will be slower and not atomic as a result.
08:27:49 dbt_clickhouse adapter: Unexpected server exception dropping table
08:27:49
08:27:49 Finished running in 0 hours 0 minutes and 2.68 seconds (2.68s).
08:27:49 Encountered an error:
Database Error
'NoneType' object has no attribute 'version_major'
make: *** [dbt-run] Error 2
```
With :
- clickhouse : 23.8.4.69
- dbt-clickhouse : 1.7.7
- dbt-core : 1.7.17
While the same code is running perfectly in Linux distributions.
### Steps to reproduce
1. Get a MacOS Sonoma 14.6.1
2. Create a DBT project
3. Create a clickhouse-server instance with version 23.8.4.69
4. Create a connection profile for this instance called "my_profile"
5. Create a dummy model called "my_mart" in the DBT project
6. Execute `dbt run --target=my_profile --select=my_mart`
### Expected behaviour
A correct DBT materialization
### Code examples, such as models or profile settings
profiles.yml
```
clickhouse-dev:
type: clickhouse
driver: native
secure: true
verify: false
schema: dbt
retries: 3
connect_timeout: 100
send_receive_timeout: 3000
sync_request_timeout: 15
threads: 4
host: localhost # SSH tunneling
port: 9430 # ssh port mapping
user: user
password: password
custom_settings: {max_bytes_before_external_group_by: '68719480000', max_bytes_before_external_sort: '68719480000'}
```
### dbt and/or ClickHouse server logs
```
08:27:45 Running with dbt=1.7.17
08:27:46 Registered adapter: clickhouse=1.7.7
08:27:46 Found 65 models, 3 seeds, 280 tests, 85 sources, 0 exposures, 0 metrics, 942 macros, 0 groups, 0 semantic models
08:27:46
08:27:48 dbt_clickhouse adapter: ClickHouse server does not support the EXCHANGE TABLES command
08:27:48 dbt_clickhouse adapter: This can be caused by an obsolete ClickHouse version or by running ClickHouse on
08:27:48 dbt_clickhouse adapter: an operating system that does not support the low level renameat2() system call.
08:27:48 dbt_clickhouse adapter: Some DBT materializations will be slower and not atomic as a result.
08:27:49 dbt_clickhouse adapter: Unexpected server exception dropping table
08:27:49
08:27:49 Finished running in 0 hours 0 minutes and 2.68 seconds (2.68s).
08:27:49 Encountered an error:
Database Error
'NoneType' object has no attribute 'version_major'
make: *** [dbt-run] Error 2
```
### Configuration
#### Environment
* dbt version: 1.7.17
* dbt-clickhouse version: 1.7.7
* clickhouse-driver version (if using native): 0.2.7
* clickhouse-connect version (if using http): 0.7.11
* Python version: 3.11
* Operating system: Sonoma 14.6.1
#### ClickHouse server
* ClickHouse Server version: 23.8.4.69
Contributor guide
Research direction
Start by reproducing the reported `dbt run --target=my_profile --select=my_mart` failure on macOS with ClickHouse 23.8.4.69, then trace the EXCHANGE TABLES fallback and the `'NoneType' object has no attribute 'version_major'` error. Done means the dummy model receives a correct DBT materialization on macOS without the reported exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100