pingcap / pingcap/tidb

br inspector tool

Open
#67,311 0 comments 0 reactions 0 assignees View on GitHub
contribution type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Think of low-level inspector which allows to analyze PiTR logs, to extract important statistics for perf / correctness fixes. I've build such tool and it helps a lot.

Sample output:

#### Example usage

```
$ ./inspector metakv \
-s s3://bucket/path/to/log-backup \
--start-ts 465149443263692803 \
--restored-ts 465149600550092800

=== PiTR Meta KV Entry Analysis ===

TS Range: [465149443263692803, 465149600550092800]
Data files processed: 50
Parse errors: 0

--- Top-Level Key Breakdown ---
Category | CF | Entries | Key Bytes | Value Bytes | Unique Keys | Avg Versions
-----------------------------------------------------------------------------------------------
mDB | write | 182,580 | 9.91 MB | 5.39 MB | 47 | 3,884

--- mDB Breakdown by Field Type (top 20) ---
DB ID | Table ID | Type | CF | Entries | Key Bytes | Value Bytes
--------------------------------------------------------------------------------
66,915 | 66,925 | IID | write | 67,463 | 3.99 MB | 1.99 MB
66,915 | 66,949 | IID | write | 54,399 | 3.22 MB | 1.61 MB
80 | 801 | IID | write | 27,617 | 1.16 MB | 836.06 KB
80 | 12,525 | IID | write | 9,420 | 487.56 KB | 285.18 KB
80 | 757 | IID | write | 7,035 | 302.29 KB | 212.97 KB
... (truncated)

--- mDB Summary by Field Type ---
Type | CF | Entries | % of mDB
--------------------------------------------------
IID | write | 176,941 | 96.9%
TID | write | 5,639 | 3.1%
```

```
$ ./inspector backupmeta \
-s s3://bucket/path/to/log-backup \
--start-ts 465149443263692803 \
--restored-ts 465149600550092800

=== PiTR Log Backup Metadata Inspection ===

TS Range: [465149443263692803, 465149600550092800]
Metadata files scanned: 715
Total file groups: 767

--- Total Data Size ---
7.83 GB (8405271070 bytes)

--- Data Size Per Store ---
Store 1: 21.63 MB (22681854 bytes)
Store 4: 30.20 MB (31667761 bytes)
Store 5: 28.10 MB (29466345 bytes)
... (truncated)

--- Metadata File Size Per Store ---
Store 1: 580.10 KB (594027 bytes)
Store 4: 645.46 KB (660950 bytes)
Store 5: 602.83 KB (617297 bytes)
... (truncated)
Total: 82.39 MB (86392101 bytes)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.