cockroachdb / cockroachdb/pebble
db: download compactions are being counted towards the compaction concurrency
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
We add download compactions to the `inProgress` compaction map here https://github.com/cockroachdb/pebble/blob/master/download.go#L439 (we also add flushes to this map). Download compactions have their own concurrency limit specified by `Options.MaxConcurrentDownloads()`. For this reason these download compactions don't increment `DB.mu.compact.compactingCount`, and instead increment `DB.mu.compact.downloadingCount`. But in `compactionPickerByScore.pickAuto` we are using the length of the in-progress compactions (which excludes flushes but includes download compactions) to decide whether we are allowed to pick another compaction. https://github.com/cockroachdb/pebble/blob/1157615755bcdd234268c712fe7c540ae7069a6d/compaction_picker.go#L1172
This is incorrect -- download compactions should be excluded from this length calculation.
Jira issue: PEBBLE-336
Epic CRDB-50824
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.