roachtest/mixedversion: failure injection

### Background

The mixedversion framework uses mutators [1] to inject (external) cluster changes to an otherwise idealized upgrade test plan. The two most common mutators are `preserve_downgrade_option_randomizer` and `cluster_setting[$name]`. The former is used to emulate the user action of disabling auto-upgrade, at any point during the upgrade plan. The latter is used to emulate updating (setting or unsetting) a cluster setting. Thus, mutators enable metamorphic testing of upgrade plans.

While the existing mutators emulate some realistic user actions, they can be seen as mostly _altruistic_ rather than _antagonistic_. That is, they do not _directly_ inject failure. (Indirectly, forgetting to set `clusterSettingMinimumVersion` could induce a failure if the cluster setting isn't available in earlier versions.) Without failure injection, the upgrade plan is still rather idealized–it doesn't test recovery (code) paths or code paths which are forced during transient (node) unavailability. Thus, by injecting recoverable failure(s), we hope to expand the overall test coverage around upgrade plans.

### Goal

Our goal is to enable a _wide_ range of failure scenarios by leveraging the failure injection framework [2]. An example of a `panic` mutator can be seen in [3]. Even panicking a node and then restarting it isn't trivial. The orchestration has to be careful to preserve the initial `--join` target and avoid any races; e.g., starting while process is still exiting. Further, the `panic` mutator is virtually devoid of dynamic preconditions. E.g., a replication factor of 5 permits up to 2 simultaneous node failures, whereas the `panic` mutator assumes only one node failure is recoverable.

By leveraging the failure injection framework, we can delegate most of the complexity around (failure) orchestration to the framework while focusing on the test mutator to ensure the resulting upgrade plan is _sound_ and _diverse_. Soundness implies that the upgrade plan doesn't violate any (global) availability constraints; i.e., only recoverable failures are injected. Since we can't really hope for _completeness_, diversity is the next best thing-the ability to cover a wide spectrum of recoverably failure scenarios.

[1] https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/roachtestutil/mixedversion/README.md#mutators
[2] https://github.com/cockroachdb/cockroach/blob/027fb22e5bb90ddaee4b6b254e9cfe523ad30ad2/pkg/roachprod/failureinjection/failures/failure.go#L43-L65
[3] https://github.com/cockroachdb/cockroach/pull/147641