test deadline in #124 is too quick (10 seconds) for busy machine?

🤖 Ran into this failing test (from #124) when running the file locally, but one step earlier than the original report — at the readiness poll rather than at the assertion:

[ Info: isready timed out on attempt 100 (max 100)
Test Failed at test/distributed_exec.jl:230
  Expression: w == :ok
   Evaluated: :timed_out == :ok

So timedwait(() -> isready(fetch(x)), 10) on wid2 gave up after 10s on attempt 100 of 100 (1–99 were clean). #124 itself failed at @test testval == 1 on line 252.

I don't think this is the same race, because the conditions were quite different:

• single-threaded (Threads.nthreads() == 1), not -t4,4
• Julia 1.14.0-DEV.2336, x86_64-linux-gnu
• 128 CPUs at a load average of ~170 — I was running the suite repeatedly on a saturated machine
• my tree had unrelated local changes (message-framing helpers factored out of send_msg_/message_handler_loop, launch(::LocalManager) split up); nothing near start_gc_msgs_task, flush_gc_msgs, or the Future/client-ref paths

With a single thread, Threads.@spawn and @async can't interleave the add_client/del_client flush the way the original race needed, so a 10s deadline missed under load looks far more likely than a resurgence. For the record, start_gc_msgs_task is still @async on master, and the only remaining Threads.@spawns are in workerpool.jl.

If that reading is right, the thing to adjust is the guard rather than the code under test. Lines 227–230 exist only to stabilize the real check that follows, so @test-ing a fixed 10s deadline turns a slow machine into a red suite for a reason unrelated to what the test is about. Either raising the deadline (it only costs wall time when something is genuinely wrong) or dropping the @test and letting the following assertion speak would avoid that; blocking on the future instead of polling would too, but it would hang the job rather than fail it if the race ever does come back.

Start with test/distributed_exec.jl around lines 227–230 and run the distributed execution test on a busy machine. Review the readiness polling guard and choose an adjustment that avoids false failures from the fixed 10-second deadline without masking the real assertion that follows. Done means the test remains meaningful and no longer flakes solely because the machine is slow.