lightning ingesting SST will interfere the its SQL action
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
Though the full-speed (import mode is on) lightning import does not care about the SQL workload, lightning internally will use SQL to implement some functionality. When the ingested SST files overlap too severe (for example, multiple lightning instances are importing or lightning disk quota feature is on), lightning's SQL-based functionality is interfered by itself.
Case 1:
After lightning finish import, it will update checkpoint and metdata table. But the overlapped SST files causes a huge compaction at TiKV side and SQL interface is throttled. I have seen a 40 minutes SQL just want to insert one row to lightning's metadata table.
Case 2:
When lightning start importing an engine sometimes it will check the metadata table like https://github.com/pingcap/tidb/blob/f5ac38d3fc15eb7c3d6c1080981b4af8a2a6eff3/br/pkg/lightning/importer/table_import.go#L1226C32-L1226C32 , and this SQL may fail due to pessimistic transaction lock timeout
Contributor guide
Assessment
This issue has not been assessed yet.