cockroachdb / cockroachdb/cockroach
storage: investigate loss of efficiency during some sequential IMPORTs
Open
A-storage
branch-master
C-bug
T-storage
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
IMPORTing from a sequential data set is generally much more efficient, because non-overlapping sstables are ingested into the storage engine. This avoids unnecessary compactions, allowing data to slot low into the LSM immediately.
Sometimes we've observed sequential IMPORTs that for some reason experience an order of magnitude worse compaction activity, and the IMPORT performance tanks. See https://github.com/cockroachdb/cockroach/issues/156187#issuecomment-3469301171
I wonder if some unlucky range operations result in some LSM structure that causes things to degrade.
Jira issue: CRDB-56059
Contributor guide
Assessment
This issue has not been assessed yet.