pingcap / pingcap/tidb

column masking policy: define snapshot/stale-read semantics and strengthen test coverage

Open
#67,272 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

# Issue Draft: Column Masking Policy Snapshot/Stale-Read Compatibility

## Title
Column masking policy: define snapshot/stale-read semantics and strengthen compatibility test coverage

## Background

Column masking policy has hybrid characteristics:

- Like privilege metadata: policy rows are stored outside table meta, with `mysql.tidb_masking_policy` as SSOT.
- Like schema object semantics: policy changes are expected to be visible with schema-level correctness expectations.

Current design already adopts delayed loading of masking policy metadata from `mysql.tidb_masking_policy` during InfoSchema access.

## Problem Statement

For historical reads, we need an explicit compatibility contract:

- `SET @@tidb_snapshot`
- `SELECT ... AS OF TIMESTAMP ...`
- `SET TRANSACTION READ ONLY AS OF TIMESTAMP ...`
- `START TRANSACTION READ ONLY AS OF TIMESTAMP ...`
- `@@tidb_read_staleness`

When policy metadata is loaded lazily via internal SQL, policy resolution must be consistent with the statement/transaction read timestamp, not accidentally mixed with latest metadata.

Even if current behavior is partially correct, this area currently lacks explicit semantic contract and dedicated end-to-end test matrix.

## Expected Semantic Contract

1. For any statement executed at read timestamp `T`, effective masking policy should be the policy state visible at `T`.
2. Policy visibility should be snapshot-consistent with table schema visibility at `T`.
3. No cross-time mix is allowed (e.g. historical table schema + latest policy metadata).
4. If policy did not exist at `T`, masking should not be applied.
5. If policy existed at `T` and was enabled, masking must be applied at `T` semantics.

## Coverage Gaps to Fill

Add explicit tests for policy history across timestamps:

- Create policy at `t1`, alter expression at `t2`, disable/enable at `t3/t4`, drop at `t5`.
- Verify query behavior at snapshots `t0=t5`.
- Cover each stale/snapshot entry path:
- `tidb_snapshot`
- statement-level `AS OF TIMESTAMP`
- `SET TRANSACTION READ ONLY AS OF TIMESTAMP`
- `START TRANSACTION READ ONLY AS OF TIMESTAMP`
- `tidb_read_staleness`
- Cover both non-prepared and prepared statement paths.
- Cover cross-session behavior to ensure no stale cache leakage across sessions.

## Suggested Test Placement

- Integration-style policy behavior:
- `tests/integrationtest/t/privilege/column_masking_policy.test`
- Engine/stale-read mechanism coverage:
- `pkg/executor/stale_txn_test.go`
- or dedicated masking+stale-read test file under `pkg/planner/core`/`pkg/executor`

## Acceptance Criteria

- A documented semantic contract (design/spec text) for masking policy under snapshot/stale-read.
- End-to-end tests proving timestamp-consistent masking behavior for all supported stale-read entry paths.
- No regression in existing masking policy and stale-read test suites.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.