ClickHouse / ClickHouse/ClickHouse

query_masking_rules are not applied to distributed_ddl_queue

Open
#56,860 0 comments 0 reactions 0 assignees View on GitHub
comp-ddl enhancement external
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
20h 33m
Merged PRs (30d)
501

Description

## Setup
Clickhouse version: 23.8

## What is wrong

`query_masking_rules` are generally used to remove secrets from logs.
It is working for `query_log`. However, `distributed_ddl_queue.query` was added and the masking is not applied to the new column.

## How to reproduce

1. Create config with a masking rule
```


postgres_table_engine
(CREATE\s+TABLE\s+.*PostgreSQL).*
"\1************'"

```
2. create cluster `default`
3. execute `CREATE TABLE test_replicas on cluster default (id UInt32, name String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')`

### Results
```
> select query from system.query_log;
CREATE TABLE test_replicas ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL************
> select query from system.distributed_ddl_queue;
"CREATE TABLE default.test_replicas UUID '619e6d16-d660-4367-b8e3-0e456f7b660e' ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')", "CREATE TABLE default.test_replicas UUID '619e6d16-d660-4367-b8e3-0e456f7b660e' ON CLUSTER rd (`id` UInt32, `name` String) ENGINE = PostgreSQL('postgres:5432', 'clickhouse', 'test_replicas', 'postgres', 'mysecretpassword')"
```
## Expected behavior
No secret leakage, masking rules applied everywhere
1. `distributed_ddl_queue` (fix needed)
2. `query_log` (already ok)
3. `asynchronous_inserts` (probably cannot contain secrets, but can in the future)
4. something else?

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the supplied query_masking_rules configuration and distributed DDL statement. Trace the query_masking_rules handling and the distributed_ddl_queue query entry point, then verify that the stored query has no secret while query_log behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases, distributed-systems, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.