pingcap / pingcap/tidb

collect table statistics when importing data by Lightning

Open
#40,469 1 comment 0 reactions 0 assignees View on GitHub
component/lightning component/statistics sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Currently, after we import data to the cluster, we need to analyze the table, which is time-consuming since it needs to scan the whole table. Collecting table statistics can be done when iterating all the data. There are several steps in Lightning that iterate all the data, such as scanning CSV files, passing KV values streamingly from Lightning to TiKV, and checksum. Hence collecting table statistics can be done in any of those steps to avoid another table full scan.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.