Random test failure in remake 4.3: output-sync deadlocks sometimes

Hi. I'm the Debian maintainer for remake. This bug report just came in to the Debian BTS:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1108812

This is on a clean Debian GNU/Linux machine on amd64. The reporter is observing the output-sync test fail. Sometimes. I poked around, and I see this as well, on his machine. What is supposed to happen is (from "tests/scripts/features/output-sync"):

#   foo/Makefile - has a "foo" target that waits for the bar target
#   bar/Makefile - has a "bar" target that runs immediately
#                - has a "baz" target that waits for the foo target

What happens when the test fails is that the "bar" target builds its prerequisite, but the recipe for "bar" itself is never run, so the "foo" and "baz" targets never complete, because "bar" never runs. I looked at various diagnostics, but cannot see why this is. We run "remake -j" so all the jobs should be queued. The logs (-x and -d) don't have any errors, the expected job just doesn't materialize.

I cannot reproduce this on my machine, and I cannot see this with make 4.3 either.

I'm attaching logs of "remake -x -d -j -Orecurse" for good and bad runs. The good run eventually does

Must remake target 'bar'

while the bad run does not. Any debugging suggestions? The reported might be willing to give you access to the failing box, if that is helpful. Thanks

log.bad.server.txt

log.good.txt

Start with tests/scripts/features/output-sync and compare the attached good and bad logs while running remake -x -d -j -Orecurse. Trace why the bar recipe is sometimes not materialized after its prerequisite; done means the output-sync test completes reliably and Must remake target 'bar' appears in the failing scenario.