pingcap / pingcap/tidb

When there're many slow log files, reading slow log is slow

Open
#65,401 15 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Seems https://github.com/pingcap/tidb/blob/c342953b2d3f52310593bd6cb2a13ea93ceed8fd/pkg/executor/slow_query.go#L944 read slow log files one-by-one. And in a user scenario of 1000 file * 300MB per file, reading slow log will cost 2 seconds. In some cases of cluster-level query, it will totally cost 2 minutes.

No matter what property the upper layer needs, we still can eagerly and concurrently open files and check their contents.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.