crosscluster: waitUntilReplicatedTime performance varies significantly in unit tests

The runtime of the e2e PCR tests (in replication_stream_e2e_test.go or stream_ingestion_job_test.go, for example) can vary significantly from 2 or 3 seconds to almost 10 seconds or even 30 seconds, due to the runtime of the WaitUntilReplicatedTime function. We should take a stab at reducing time it takes to wait for the replicated time.

One theory:
- c2c creates a lot of goroutines running on a single test server. perhaps elastic cpu admission control is bogging our unit test perf down when it detects goroutine scheduling latency.

Theories ruled out:
- we recently bumped the the number of ingestion processors running per node, but in these unit tests, we still only schedule one processor per node.
- reducing `physical_replication.producer.timestamp_granularity` to 100ms

This problem likely occurs in LDR as well.

Jira issue: CRDB-42855