cockroachdb / cockroachdb/pebble

db: download compactions are being counted towards the compaction concurrency

Open
#4,299 0 comments 0 reactions 1 assignee Claimed by @RaduBerinde View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

We add download compactions to the `inProgress` compaction map here https://github.com/cockroachdb/pebble/blob/master/download.go#L439 (we also add flushes to this map). Download compactions have their own concurrency limit specified by `Options.MaxConcurrentDownloads()`. For this reason these download compactions don't increment `DB.mu.compact.compactingCount`, and instead increment `DB.mu.compact.downloadingCount`. But in `compactionPickerByScore.pickAuto` we are using the length of the in-progress compactions (which excludes flushes but includes download compactions) to decide whether we are allowed to pick another compaction. https://github.com/cockroachdb/pebble/blob/1157615755bcdd234268c712fe7c540ae7069a6d/compaction_picker.go#L1172

This is incorrect -- download compactions should be excluded from this length calculation.

Jira issue: PEBBLE-336

Epic CRDB-50824

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.