When there're many slow log files, reading slow log is slow
Open
type/enhancement
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
Seems https://github.com/pingcap/tidb/blob/c342953b2d3f52310593bd6cb2a13ea93ceed8fd/pkg/executor/slow_query.go#L944 read slow log files one-by-one. And in a user scenario of 1000 file * 300MB per file, reading slow log will cost 2 seconds. In some cases of cluster-level query, it will totally cost 2 minutes.
No matter what property the upper layer needs, we still can eagerly and concurrently open files and check their contents.
Contributor guide
Assessment
This issue has not been assessed yet.