cockroachdb / cockroachdb/cockroach

operations: support background execution of long-running operations

Open
#166,412 1 comment 0 reactions 1 assignee Claimed by @shailendra-patel View on GitHub
C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Some operations (e.g., `backup-restore`, `add-index`, `manual-compaction`) run for hours and block a regular worker for the entire duration, or time out before completing. These operations reduce runner throughput by occupying limited worker slots.

**Proposal:** Add a `LongRunning bool` field to `OperationSpec`. Operations marked as long-running are executed by a separate pool of background workers with a configurable concurrency limit (e.g., `--long-running-operation-parallelism`). The operation implementation remains unchanged — it runs the same `Run()` and returns cleanup as usual. The only difference is scheduling: long-running operations don't consume regular worker slots.

This keeps the regular worker pool available for short, frequent operations while giving long-running operations dedicated capacity.

**Key files:** `pkg/cmd/roachtest/run_operation.go`, `pkg/cmd/roachtest/registry/operation_spec.go`

Jira issue: CRDB-61915

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.