roachtest: restore/online-restore-chaos/tpcc-5k/gce/nodes=4/cpus=8 failed

**Note:** This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.restore/online-restore-chaos/tpcc-5k/gce/nodes=4/cpus=8 [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21508672?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21508672?buildTab=artifacts#/restore/online-restore-chaos/tpcc-5k/gce/nodes=4/cpus=8) on master @ [5318641d2b0ff04dbdaced9f83ecb0d692f697b6](https://github.com/cockroachdb/cockroach/commits/5318641d2b0ff04dbdaced9f83ecb0d692f697b6):

Failed with:

```
(assertions.go:387).Fail:
Error Trace: pkg/cmd/roachtest/tests/backup_restore_roundtrip.go:476
pkg/cmd/roachtest/tests/online_restore.go:580
pkg/cmd/roachtest/tests/online_restore.go:515
pkg/cmd/roachtest/test_runner.go:1992
src/runtime/asm_amd64.s:1771
Error: Received unexpected error:
job 1209081807922987010 unexpectedly in state pause-requested while tracking progress
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*CommonTestUtils).waitForJobFractionCompletedWithNode
| pkg/cmd/roachtest/tests/backup_restore_utils.go:594
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.injectAndRecoverFailure.func1
| pkg/cmd/roachtest/tests/backup_restore_roundtrip.go:451
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/roachtestutil/task.(*group).GoWithCancel.func1
| pkg/cmd/roachtest/roachtestutil/task/manager.go:201
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/roachtestutil/task.(*group).GoWithCancel.func2
| pkg/cmd/roachtest/roachtestutil/task/manager.go:211
| golang.org/x/sync/errgroup.(*Group).Go.func1
| external/org_golang_x_sync/errgroup/errgroup.go:93
| runtime.goexit
| src/runtime/asm_amd64.s:1771
Wraps: (2) job 1209081807922987010 unexpectedly in state pause-requested while tracking progress
Error types: (1) *withstack.withStack (2) *errutil.leafError
Test: restore/online-restore-chaos/tpcc-5k/gce/nodes=4/cpus=8
(require.go:1401).NoError: FailNow called
test artifacts and logs in: /artifacts/restore/online-restore-chaos/tpcc-5k/gce/nodes=4/cpus=8/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21508672-1789019633-188-n5cpu8-0001 | | 10.91.97.125 |
| teamcity-21508672-1789019633-188-n5cpu8-0002 | | 10.91.97.110 |
| teamcity-21508672-1789019633-188-n5cpu8-0003 | | 10.91.97.144 |
| teamcity-21508672-1789019633-188-n5cpu8-0004 | | 10.91.97.166 |
| teamcity-21508672-1789019633-188-n5cpu8-0005 | | 10.91.97.222 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=8
- diskCount=0
- encrypted=true
- fs=ext4
- localSSD=false
- runtimeAssertionsBuild=true
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21508672/restore-online-restore-chaos-tpcc-5k-gce-nodes-4-cpus-8/1789052858187/1789058595970)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21508672-1789019633-188-n5cpu8&from_ts=1789052858187&to_ts=1789058595970&live=false&storage=flex_tier)

/cc @cockroachlabs/disaster-recovery

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68165