roachtest: nightly suite runner is extremely overloaded, causes test failures

For a long time, we've had issues with tests in the roachtest nightly suite failing in a mysterious way that suggested that a `time.Sleep(time.Second)` was sometimes taking much (>>20x) longer.

I finally got to the bottom of it and the answer appears to be that the roachtest runner machine is severely overloaded: CPU utilization is at 100%, the machine is almost out of memory, and heap GC runs take north of a minute.

It's surprising that anything works at all under these circumstances, but I'm sure it causes all kinds of spurious failures across many tests that we likely shrug off as "infra flakes".

We need to find out why this machine is so overloaded, fix that, and set up monitoring for the health of the roachtest machine.

https://github.com/cockroachdb/cockroach/issues/138864 has the in-depth analysis. The execution trace referred in that analysis is in [this slack thread] (in case the artifacts are already gone).

[this slack thread]: https://cockroachlabs.slack.com/archives/C06KHP4PU4X/p1736761518749049?thread_ts=1736204265.030069&cid=C06KHP4PU4X

Jira issue: CRDB-46415