Test flakes with Real Temporal Server leading to infinite activity retry

io.temporal.internal.testing.WorkflowTestingTest > testActivityFailure

fails with

io.temporal.internal.testing.WorkflowTestingTest > testActivityFailure STANDARD_OUT

    21:11:19.001 [Activity Executor taskQueue="test-workflow", namespace="default": 1] WARN  i.t.i.a.POJOActivityTaskHandler - Activity failure. ActivityId=38334385-1f57-30b4-bc10-97377255f018, activityType=Execute, attempt=1

    java.lang.IllegalThreadStateException: Execute-input1
    	at io.temporal.internal.testing.WorkflowTestingTest$FailingActivityImpl.execute(WorkflowTestingTest.java:596)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214)
    	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180)
    	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120)
    	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204)
    	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:14)
    	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:750)
.......

    21:20:04.797 [Activity Executor taskQueue="test-workflow", namespace="default": 21] WARN  i.t.i.a.POJOActivityTaskHandler - Activity failure. ActivityId=38334385-1f57-30b4-bc10-97377255f018, activityType=Execute, attempt=78094

    java.lang.IllegalThreadStateException: Execute-input1
    	at io.temporal.internal.testing.WorkflowTestingTest$FailingActivityImpl.execute(WorkflowTestingTest.java:596)
    	at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214)
    	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180)
    	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120)
    	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204)
    	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164)
    	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:750)

and getting killed by timeout.

It flakes with real Temporal Server only and may manifest a severe server issue.

java-sdk-public_build_649_docker-unit-test-with-docker-service.log.zip

Start with io.temporal.internal.testing.WorkflowTestingTest, especially testActivityFailure and FailingActivityImpl.execute at WorkflowTestingTest.java:596, then reproduce it against a real Temporal Server using the linked Docker test setup. Trace why the activity reaches repeated attempts instead of completing or stopping; done means the test no longer retries indefinitely or times out.