pingcap / pingcap/tidb

[Column Masking] HIGH: "AT RESULT" semantics conflict with implementation

Open
#67,044 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Severity
HIGH

## Description
### Design Document Claims
The design document claims that JOIN/WHERE/GROUP BY/HAVING/ORDER BY/set operators all calculate based on **original values**:

> "AT RESULT: 各子句计算用原值"
> — `docs/design/2026-02-27-column-level-masking.md:66`

### Actual Implementation Behavior
The implementation does column replacement (masking) on select expressions inside `buildProjection`, but:
1. **HAVING** and **ORDER BY** are built **after** projection
2. **Set operators** (UNION/EXCEPT/INTERSECT) children's `buildSelect` also does projection masking first, then enters set operation
3. **ORDER BY** referencing a column not in select list gets appended as Auxiliary field (no masking), causing whether the column is selected to change ORDER BY's use of original vs masked value

## Impact

### 1. May Change Result Correctness/Compatibility
- HAVING filter conditions may be based on masked values instead of original values
- ORDER BY sort keys may be based on masked values instead of original values
- UNION DISTINCT deduplication keys may be based on masked values instead of original values

### 2. Inconsistent Behavior
```sql
-- Scenario 1: c is in select list, ORDER BY may use masked value
SELECT c FROM t ORDER BY c;

-- Scenario 2: c is not in select list (as auxiliary), ORDER BY uses original value
SELECT id FROM t ORDER BY c;
```

These two scenarios may produce different sort results!

## Related Code Locations
- `docs/design/2026-02-27-column-level-masking.md:66` - design semantic declaration
- `pkg/planner/core/logical_plan_builder.go:4062,4067,4114` - buildSelect: projection first, then having, then order by
- `pkg/planner/core/logical_plan_builder.go:1311,1531` - masking injected inside projection
- `pkg/planner/core/logical_plan_builder.go:2393` - ORDER BY appends auxiliary columns
- `pkg/planner/core/logical_plan_builder.go:1382` - auxiliary columns don't go through masking
- `pkg/planner/core/logical_plan_builder.go:1896` - set-op children built via buildSelect

## Proposed Fix

### Step 1: Clarify Product Semantics (Decision Required)

**Option A: Strict AT RESULT (per design document)**
- All relational operations (JOIN/WHERE/GROUP BY/HAVING/ORDER BY/set operators) use original values
- Masking only applied at final output

**Option B: Accept current implementation semantics (projection-level replacement)**
- Update design doc and test plan
- Add explicit rules and regression tests
- Acknowledge HAVING/ORDER BY/set operators may see masked values

### Step 2: Implementation Fix (if choosing Option A)
Move masking application to query-block top level (after HAVING/ORDER BY/set operators), or use raw auxiliary columns for sorting/filtering/set operations while output uses masked columns.

## Parent Issue
Relates to #65744

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.