pingcap / pingcap/tidb

[lightning] automatically choose logical or physical import mode based on the table size.

Open
#42,470 1 comment 0 reactions 0 assignees View on GitHub
component/lightning type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

**Is your feature request related to a problem? Please describe:**

The logical import mode is significantly faster than the physical mode when importing small tables and the performance reverses on large tables. When a user wants to import databases with a large number of small tables together with a few huge tables, currently users have to split those tables into 2 batches by file size and then import them in 2 executions using different import modes. I would suggest supporting the hybrid import mode so that the tidb-lightning chooses the appropriate import mode by checking the file size.

**Describe the feature you'd like:**

**Describe alternatives you've considered:**

**Teachability, Documentation, Adoption, Migration Strategy:**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.