ClickHouse / ClickHouse/dbt-clickhouse
ALTER UPDATE/ALTER DELETE statement with subquery may be nondeterministic
- Dominant language
- Python
- Stars
- 362
- Forks
- 177
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 8
Description
### Describe the bug
I know this has been discussed before but it is still an issue for us...
Frequent failures of incremental (delete+insert) models in clustered clickhouse (24.2.2.71). Re-running several times normally completes the process, however this seems to occur frequently enough to cause us difficulties.
The error is raised on the following step :
```
delete from
`dbt_prod_marts`.`mrt_my_table`
where
(
event_date, x,y,z
) in (
select
event_date,
x,y,z
from
`dbt_prod_marts`.`mrt_my_table__dbt_new_data_751e553e_44e5_4a12_99b7_fa79f323561a`
)
)
```
I think that perhaps what might be happening is that the 'temporary' table `mrt_my_table__dbt_new_data_751e553e_44e5_4a12_99b7_fa79f323561a` is not fully replicated by the time the deletion is run.
Is there a way that we can wait for replication of the 'temporary' table to complete before performing the delete step?
### Steps to reproduce
1. create cluster with 2 replica nodes
2. create delete+insert incremental model as a ReplicatedMergetree
3. Materialize the table (may need a significant amount of data and possibly multiple attempts to reproduce).
### Expected behaviour
Should robustly materialize the table
### Code examples, such as models or profile settings
I have `allow_nondeterministic_mutations` set in my user profile, but the problem still occurs.
### dbt and/or ClickHouse server logs
```
executeQuery: Code: 36. DB::Exception: ALTER UPDATE/ALTER DELETE statement with subquery may be nondeterministic, see allow_nondeterministic_mutations setting. (BAD_ARGUMENTS) (version 24.2.2.71 (official build)) (from 10.26.0.61:4171) (in query: /* {"app": "dbt", "dbt_version": "1.7.14", "profile_name": "clickhouse", "target_name": "prod", "node_id": "redacted*/
--
8. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x0000000012f672d5
9. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x0000000012f6bc10
10. DB::HTTPServerConnection::run() @ 0x0000000012fe5ffa
11. Poco::Net::TCPServerConnection::start() @ 0x00000000158bc9b2
12. Poco::Net::TCPServerDispatcher::run() @ 0x00000000158bd7b1
13. Poco::PooledThread::run() @ 0x00000000159b6387
14. Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000159b49bd
15. ? @ 0x00007deb343be609
16. ? @ 0x00007deb342e3353
```
### Configuration
#### Environment
* dbt version: 1.7.14
* dbt-clickhouse version: 1.7.6
#### ClickHouse server
* ClickHouse Server version: 24.2.2.71
Contributor guide
Research direction
No source file or test is named in the report. Start by locating the delete+insert incremental materialization path and reproduce it with a two-replica ReplicatedMergeTree cluster using the shown ALTER DELETE subquery; compare behavior while the temporary table replicates. Done means the model materializes robustly without the nondeterministic-mutations failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, python, sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100