[Enhancement] Introduce unified sensitive-field redaction for FE SQL logs
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [x] I searched the existing open and closed issues and found no similar issue.
### Description
FE diagnostic logs are produced by multiple SQL processing paths, and these paths do not currently share one consistent policy for handling sensitive property values.
Removing the complete SQL text from logs makes production troubleshooting significantly harder. The desired behavior is to preserve the SQL structure, identifiers, and other non-sensitive content while redacting only values associated with confirmed sensitive fields, such as passwords, access keys, secret keys, tokens, SASL credentials, and equivalent connector-specific properties.
This issue tracks a general FE logging improvement and hardening effort. It is not intended to describe or track a specific vulnerability.
### Proposed solution
Introduce a shared SQL log redaction facility with the following properties:
1. Preserve non-sensitive SQL content and statement structure so that logs remain useful for diagnosis.
2. Replace only the values of known sensitive fields. Reuse a central sensitive-key registry where possible and support statement-specific keys for resources, stages, routine load, catalogs, repositories, and similar property-bearing statements.
3. Provide consistent handling for:
- successfully parsed statements;
- parser-error paths;
- SQL dialect conversion paths;
- statements whose parser or plan object is unavailable.
4. Use a fail-safe fallback: if the redactor cannot safely process a statement, do not emit the original SQL from that path. Log metadata such as query ID, statement type, length, user, dialect, and error type for that event instead.
5. Migrate the main FE diagnostic logging entry points to the shared facility, including:
- StmtExecutor and ConnectProcessor;
- HTTP statement execution endpoints;
- parser and dialect-conversion logging;
- Load command logging;
- CBO/rewrite logging;
- query profile logging.
6. Keep the regular diagnostic-log policy aligned with, but clearly separated from, audit-log encryption behavior.
### Tests
Add log-capture tests that verify:
- known sensitive values never appear in emitted logs;
- SQL structure and non-sensitive content remain available;
- statement-specific sensitive keys are handled;
- parser failures and dialect-conversion failures use the safe fallback;
- all migrated entry points apply the same policy.
### Are you willing to submit PR?
- [ ] Yes, I am willing to submit a PR.
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct.
Contributor guide
Research direction
Start by mapping the listed FE diagnostic entry points, including StmtExecutor, ConnectProcessor, HTTP statement execution, parser and dialect-conversion logging, Load commands, CBO/rewrite logging, and query profiles. Then inspect the existing sensitive-key registry and the separation between diagnostic logging and audit-log encryption. Done means log-capture tests show sensitive values are absent, useful SQL content is preserved, failures use safe metadata fallback, and all listed paths share the policy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases, observability, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100