ClickHouse / ClickHouse/ClickHouse
query_masking_rules are not applied to distributed_ddl_queue
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 20h 33m
- Merged PRs (30d)
- 501
Description
## Setup
Clickhouse version: 23.8
## What is wrong
`query_masking_rules` are generally used to remove secrets from logs.
It is working for `query_log`. However, `distributed_ddl_queue.query` was added and the masking is not applied to the new column.
## How to reproduce
1. Create config with a masking rule
```
postgres_table_engine
(CREATE\s+TABLE\s+.*PostgreSQL).*
"\1************'"
```
2. create cluster `default`
3. execute `CREATE TABLE test_replicas on cluster default (id UInt32, name String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')`
### Results
```
> select query from system.query_log;
CREATE TABLE test_replicas ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL************
> select query from system.distributed_ddl_queue;
"CREATE TABLE default.test_replicas UUID '619e6d16-d660-4367-b8e3-0e456f7b660e' ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')", "CREATE TABLE default.test_replicas UUID '619e6d16-d660-4367-b8e3-0e456f7b660e' ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')"
```
## Expected behavior
No secret leakage, masking rules applied everywhere
1. `distributed_ddl_queue` (fix needed)
2. `query_log` (already ok)
3. `asynchronous_inserts` (probably cannot contain secrets, but can in the future)
4. something else?
Contributor guide
Research direction
Reproduce the issue with the supplied query_masking_rules configuration and distributed DDL statement. Trace the query_masking_rules handling and the distributed_ddl_queue query entry point, then verify that the stored query has no secret while query_log behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- databases, distributed-systems, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100