[Umbrella Issue] Many tests use Thread.sleep instead of synchronization primitives

Many of our tests simply have a `Thread.sleep(duration)` when testing multi-threaded code. This is inherently flaky.

Instead we should use synchronization classes (generally `Semaphore`) to synchronized the tests which is both faster in the general case and not flaky in the extreme case.

See for instance #1218 and #1219