cockroachdb / cockroachdb/cockroach
roachtest-operation: rewrite failure injection operations to use roachprod failure injection library
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
## Summary
Rewrite the three failure injection DRT operations (`node-kill`, `network-partition`, `disk-stall`) to use the roachprod failure injection library instead of ad-hoc shell commands. The library provides a structured `FailureMode` interface with proper state management, recovery monitoring, and cleanup safety.
The `cluster.Cluster.GetFailer()` method returns a `*failures.Failer` for any registered failure mode. The disk-stall operation already partially uses this via `roachtestutil.DiskStaller` — use that as a reference pattern.
## Task 1: Rewrite `node-kill` operations
Rewrite `pkg/cmd/roachtest/operations/node_kill.go` to use the `"process-kill"` failure mode from `pkg/roachprod/failureinjection/failures/process_kill.go`. Re-register the four existing operation variants and uncomment `registerNodeKill(r)` in `register.go`.
## Task 2: Rewrite `network-partition` operations
Rewrite `pkg/cmd/roachtest/operations/network_partition.go` to use the `"iptables-network-partition"` failure mode from `pkg/roachprod/failureinjection/failures/network_partition.go`. Keep the same two operation names.
## Task 3: Align `disk-stall` cleanup with library
`pkg/cmd/roachtest/operations/disk_stall.go` already uses `roachtestutil.DiskStaller` which wraps the library, but cleanup has manual node-restart logic that duplicates what the library provides. Remove the duplication and let the library handle recovery.
Contributor guide
Assessment
This issue has not been assessed yet.