TiDB side writes SST files instead of TiKV side
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
In normal KV transaction API, TiDB takes responsibility for encoding user data to KV pairs, and TiKV takes responsibility for persisting data in its storage engine instance which is RocksDB. When comes to data ingest, the main idea is offloading workload to TiDB or other roles and only let TiKV call RocksDB's ingest SST API.
Before this request, TiKV still uses a separate gRPC endpoint to received encoded KV pairs and build SST files. And this request aims to let TiDB build SST files and send the content of SST files to TiKV.
Pros:
- SST files are compressed, so network transmission is reduced
- CPU resources to build SST files move away from TiKV to other roles which is more stateless and scalable. Also only one unit of CPU is needed rather than 3 units because of 3 peers in raft group
- ingesting SST files are now managed by TiDB or other role, which can simplify the architecture of services like PiTR or CDC
Cons:
- The hardware resource usage pattern is changed, and because we have several complex concurrency control parameters, the new pattern may need a tuning. And if the CPU resource is enough in TiKV nodes but not enough in TiDB (or other role) nodes, because SST building is CPU-bounded, performance will drop
- we are using a non-official library rather than RocksDB to build SST files, which may introduce bugs
Contributor guide
Assessment
This issue has not been assessed yet.