cockroachdb / cockroachdb/cockroach
clusterverison,migration: prevent version finalization until one or more predicates are satisfied
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
There exist use cases where it is undesirable to tie a blocking operation / migration to an internal cluster version. For example, a Pebble migration that needs to rewrite a potentially large number of tables.
Long running version finalization may be confusing to the operator. Consider a situation where a cluster needs to update to the next major version to pick up an improvement. The cluster finalization process starts, but is seemingly blocked on some long running migration.
One alternative is to block the process of _starting_ version finalization until one or more predicates have been satisfied. This removes some confusion by surfacing the reasons as to why the version upgrade is blocked, allowing an operator to react ahead of time.
**Describe the solution you'd like**
Consider adding some way of preventing cluster version finalization until one or more predicates have been satisfied.
We'd need to ensure that whatever the predicates are, that they act like a latch, i.e. once the predicate is satisfied, the latch is closed and there is no possibility that the predicate can move to the prior state.
We'd also want to ensure observability into the exact reasons for blocking version finalization are surfaced to the operator - e.g. SQL command returns and error with a list of things to check, some way to look at some view of the predicates that are unsatisfied, etc.
**Describe alternatives you've considered**
Continue with the status quo and have long running operations tied to internal cluster versions. Perhaps we could improve observability here to indicate where in the internal cluster version sequence the finalization process is, which could avoid confusion.
**Additional context**
Related to #96819.
Check out [`PreconditionFunc`](https://github.com/cockroachdb/cockroach/blob/7e2df35a2f6bf7a859bb0539c8ca43c4e72ed260/pkg/upgrade/tenant_upgrade.go#L69-L74) here for some ideas.
Jira issue: CRDB-24975
Contributor guide
Assessment
This issue has not been assessed yet.