cockroachdb / cockroachdb/cockroach

storage: investigate loss of efficiency during some sequential IMPORTs

Open
#156,566 1 comment 0 reactions 0 assignees View on GitHub
A-storage branch-master C-bug T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

IMPORTing from a sequential data set is generally much more efficient, because non-overlapping sstables are ingested into the storage engine. This avoids unnecessary compactions, allowing data to slot low into the LSM immediately.

Sometimes we've observed sequential IMPORTs that for some reason experience an order of magnitude worse compaction activity, and the IMPORT performance tanks. See https://github.com/cockroachdb/cockroach/issues/156187#issuecomment-3469301171

I wonder if some unlucky range operations result in some LSM structure that causes things to degrade.

Jira issue: CRDB-56059

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.