ClickHouse / ClickHouse/ClickHouse

Dict SOURCE/LAYOUT block state machine missing QuotedIdentifier value handling desyncs key-value tracking

Open
#101,303 0 comments 0 reactions 1 assignee Claimed by @KochetovNicolai View on GitHub
bug comp-sql-syntax minor
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

_Found via ClickGap automated review. Please close or comment if this is incorrect or needs adjustment._

_Retrospective finding from a historical scan of [PR #99260](https://github.com/ClickHouse/ClickHouse/pull/99260) (merged 2026-03-12). Confirmed on current codebase — close with a note if already fixed._

### Describe what's wrong

When dictionary SOURCE/LAYOUT blocks contain double-quoted or backtick-quoted identifier values (e.g., TABLE "mytable" DB "mydb"), the state machine does not recognize QuotedIdentifier tokens as consumed values. This causes dict_expect_key to stay false, so the next BareWord (which should be a structural key) is treated as a value and obfuscated instead of preserved.

**Root cause:** Line 1447 condition `(token.type == TokenType::Number || token.type == TokenType::StringLiteral)` is missing `|| token.type == TokenType::QuotedIdentifier`

**Affected locations:**
- `src/Parsers/obfuscateQueries.cpp:1447` — Missing QuotedIdentifier in non-BareWord value type check

**Impact:** Dictionary definitions using double-quoted or backtick-quoted values in SOURCE blocks produce incorrectly obfuscated output where structural parameter names (DB, PORT, etc.) after quoted values are obfuscated instead of preserved, resulting in invalid SQL output.

### Does it reproduce on most recent release?

Yes — confirmed on current `master` (commit `b1e6445e6f93`).

### How to reproduce

```sql
#!/usr/bin/env bash

CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$CUR_DIR"/../shell_config.sh

obf="$CLICKHOUSE_FORMAT --obfuscate"

# Test 1: DB key after double-quoted TABLE value
result=$(echo 'CREATE DICTIONARY d (id UInt64) PRIMARY KEY id SOURCE(CLICKHOUSE(TABLE "mytable" DB "mydb")) LAYOUT(FLAT()) LIFETIME(0)' | $obf)
if echo "$result" | grep -qP 'TABLE "[^"]*" DB "[^"]*"'; then
echo "OK: DB key preserved after double-quoted value"
else
echo "BUG: DB key obfuscated after double-quoted value"
fi

# Test 2: Backtick-quoted values
result=$(echo 'CREATE DICTIONARY d (id UInt64) PRIMARY KEY id SOURCE(CLICKHOUSE(TABLE `mytable` DB `mydb`)) LAYOUT(FLAT()) LIFETIME(0)' | $obf)
if echo "$result" | grep -qP 'TABLE `[^`]*` DB `[^`]*`'; then
echo "OK: DB key preserved after backtick-quoted value"
else
echo "BUG: DB key obfuscated after backtick-quoted value"
fi

# Test 3: Baseline - single-quoted values work correctly
result=$(echo "CREATE DICTIONARY d (id UInt64) PRIMARY KEY id SOURCE(CLICKHOUSE(TABLE 'mytable' DB 'mydb')) LAYOUT(FLAT()) LIFETIME(0)" | $obf)
if echo "$result" | grep -qP "TABLE '[^']*' DB '[^']*'"; then
echo "OK: DB key preserved after single-quoted value (baseline)"
else
echo "UNEXPECTED: DB key obfuscated after single-quoted value"
fi
```

### Expected behavior

```
OK: DB key preserved after double-quoted value
OK: DB key preserved after backtick-quoted value
OK: DB key preserved after single-quoted value (baseline)
```

### Error message and/or stacktrace

```
BUG: DB key obfuscated after double-quoted value
BUG: DB key obfuscated after backtick-quoted value
OK: DB key preserved after single-quoted value (baseline)
```

### Additional context

**Open risks:**
- DollarSignedStringLiteral tokens might have the same issue if used as dict SOURCE values, though this is extremely unlikely in practice.

**Suggested fix:** Add `|| token.type == TokenType::QuotedIdentifier` to the condition at line 1447: `(token.type == TokenType::Number || token.type == TokenType::StringLiteral || token.type == TokenType::QuotedIdentifier)`

**Analysis details:** Confidence HIGH | Severity P2 | Testability: `STATELESS_SQL`

Found during automated review of [PR #99260](https://github.com/ClickHouse/ClickHouse/pull/99260).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.