apple / apple/foundationdb

BulkLoad should check key/value size before loading it

Open
#12,303 0 comments 0 reactions 1 assignee Claimed by @saintstack View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

See https://forums.foundationdb.org/t/bulkdump-failed/5016?u=saintstack

Scenario: Create SST files including oversized key/value manually. Then, bulkload the SST files. Then, bulkdump the data.

Since bulkload bypasses the transaction system, the bulkload can directly write oversized key/value to the database. This is particularly dangerous in the presence of adversarial SSTs.

BulkLoad mechanism must perform SST file content check when the SST file pass across the boundary of FDB cluster.

Ref: FDB only allows key with the max size `KEY_SIZE_LIMIT` and value with the max size `VALUE_SIZE_LIMIT`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.