apache / apache/kvrocks

Add support of bulk load for the string like HBase bulkload

Open
#1,301 19 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
4.4k
Forks
658
Avg merge
1d 20h
Merged PRs (30d)
10

Description

### Motivation

Many scenarios need to bulk-load mass data regularly, and it may bring heavy workload and latency spike if loads through the API interface. So it will be better if we can offer a way to mitigate this issue.

### Solution

We can use [RocksDB Ingest SST](https://github.com/EighteenZi/rocksdb_wiki/blob/master/Creating-and-Ingesting-SST-files.md) to bulk load those data and support for simple strings only.

see more discussions in https://github.com/apache/kvrocks/discussions/1628

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked RocksDB Ingest SST documentation and the discussion at apache/kvrocks#1628 to understand the proposed bulk-load workflow. Trace how simple string data currently enters Kvrocks through its API, then define completion as a supported bulk-load path that uses SST ingestion while reducing the workload and latency of repeated mass loads.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, redis
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.