Improve Slow Log Performance and Observability
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
### Summary
Enhance the performance and observability of the TiDB slow log subsystem, including formatting, rule matching, field collection, and metrics exposure.
### Motivation
In high-concurrency or complex rule configurations, slow log formatting, rule matching, and field collection can introduce noticeable CPU and memory overhead.
At the same time, the lack of key metrics makes it difficult to evaluate the internal overhead of the slow log system itself, as well as understand dropped or skipped slow logs.
### Goals
- Reduce CPU and memory overhead in the slow log pipeline to avoid impacting normal SQL execution.
- Improve the observability of the slow log system to support online monitoring, diagnosis, and tuning.
### Tasks
- **Optimize slow log formatting**
- Currently, each slow query triggers string concatenation and formatting.
- This introduces significant overhead under high concurrency.
- **Improve metrics and observability**
- Add missing key metrics, such as:
- Slow log write rate
- Rule matching statistics
- Dropped or skipped slow log counts
- These metrics will help evaluate system overhead and diagnose slow log behavior.
- **...**
Contributor guide
Assessment
This issue has not been assessed yet.