Retain original statement summary samples when persistence is enabled and redact file output
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Feature Request
**Is your feature request related to a problem? Please describe:**
Statement summary currently applies `tidb_redact_log` when collecting `QUERY_SAMPLE_TEXT`. Enabling log redaction therefore also removes or marks the in-memory sample shown by statement-summary queries and TiDB Dashboard, even when the operator only wants to control persisted logs. The background writer also preserves the mode from collection instead of applying the setting in effect when it writes the file.
Observed in the source at TiDB master `5acf6574288567bb473762e42313e25880c02419`: `ExecStmt.GetOriginalSQL` formats using the session mode, while statement-log serialization writes `StmtRecord.SampleSQL` unchanged.
**Describe the feature you'd like:**
When the existing read-only instance setting `tidb_stmt_summary_enable_persistent` is ON, keep the original statement sample in memory (retaining sensitive-statement protections and sample-length limits), and apply the current GLOBAL `tidb_redact_log` when serializing the sample into the persistent statement log. When persistence is OFF, retain the existing collection-time redaction behavior. Do not add a separate timing variable. Persistent output modes are:
- OFF: original sample.
- ON: the already captured normalized SQL, without literal values or prepared arguments.
- MARKER: the sample with the existing redaction marker escaping.
Apply the same behavior to window rotation, shutdown/flush, and evicted-record logging without modifying records held by memory readers. For example, a sample captured with OFF and persisted after switching to ON should remain original in memory and be normalized in the file.
**Describe alternatives you've considered:**
Always keeping collection-time redaction prevents retrieving original samples in persistent deployments. Always retaining raw samples would change non-persistent deployments. A separate timing variable adds an unnecessary configuration choice when the existing persistence setting can select the behavior. Reparsing SQL while writing is unreliable because samples can be truncated or include a prepared-argument suffix; reuse the existing normalized SQL instead.
**Teachability, Documentation, Adoption, Migration Strategy:**
This changes in-memory `QUERY_SAMPLE_TEXT` only when persistence is enabled. Persistence remains disabled by default, preserving existing behavior for non-persistent deployments. The existing instance setting is read-only from SQL. History tables combine memory and persisted rows and can therefore contain both original and redacted samples. Access privileges stay unchanged. Existing log files are not rewritten. MARKER is reversible marking, not removal of literal values. This request does not change PLAN, PREV_SAMPLE_TEXT, or slow-log formatting.
Contributor guide
Research direction
Start at ExecStmt.GetOriginalSQL and the statement-log serialization that writes StmtRecord.SampleSQL, then trace the persistent writer through window rotation, shutdown/flush, and evicted-record logging. Verify the existing sensitive-statement and sample-length protections and reuse the captured normalized SQL rather than reparsing. Done means memory readers retain the specified sample while persistent output follows the current GLOBAL tidb_redact_log mode in every write path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100