Flaky test: RemotingSpec "should resume the outbound reader when passive read handoff fails" on Scala 3
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 211
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 89
Description
### Description
The test `RemotingSpec` — `"should resume the outbound reader when passive read handoff fails"` intermittently times out on the Scala 3 CI job.
### Failure
```
java.lang.AssertionError: assertion failed: timeout (6 seconds) during expectMsg while waiting for after-handoff-failure
at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
at org.apache.pekko.testkit.TestKitBase.expectMsg_internal(TestKit.scala:471)
at org.apache.pekko.testkit.TestKitBase.expectMsg(TestKit.scala:448)
at org.apache.pekko.remote.classic.RemotingSpec.f$proxy19$1(RemotingSpec.scala:797)
```
### Location
`remote/src/test/scala/org/apache/pekko/remote/classic/RemotingSpec.scala:797`
```scala
outboundRemoteHandle.association.write(payload("after-handoff-failure"))
receiverProbe.expectMsg("after-handoff-failure") // <-- times out here
```
### Observed in
- PR #3095 (unrelated changes — `Source.queue` deprecation): [CI run](https://github.com/apache/pekko/actions/runs/28965711798/job/85948213045)
- The test was added by PR #3205
### Analysis
The test sets up a passive read-only inbound association, sends a broken packet to trigger disassociation of the read-only endpoint, then expects the outbound writer to resume delivering messages. The `expectMsg` at line 797 times out, suggesting the outbound reader did not resume after the passive read handoff failure.
This is a timing-sensitive test involving multiple transport associations and handoffs. The 6-second timeout may be insufficient under CI load, or the resume logic may have a race condition that manifests more often on Scala 3 (possibly due to different compilation/optimization characteristics affecting thread scheduling).
### Possible fixes
1. Increase the `expectMsg` timeout with a `dilated` factor
2. Investigate whether the resume logic in the endpoint reader has a race condition
3. Add retry or `awaitAssert` around the assertion to tolerate transient timing
Contributor guide
Research direction
Start with remote/src/test/scala/org/apache/pekko/remote/classic/RemotingSpec.scala around line 797 and reproduce the named test on the Scala 3 CI job. Trace the passive read handoff and outbound reader behavior after the broken packet, then determine whether the timeout or resume logic is responsible. Done means the test reliably delivers "after-handoff-failure" without masking a real failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100