apache / apache/pekko-connectors

Unix Domain Socket: large transfers stall - the disabled UnixDomainSocketSpec throughput test never completes

Open
#1,858 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
82
Forks
50
Avg merge
1d 2h
Merged PRs (30d)
78

Description

## Summary

`UnixDomainSocketSpec` has a disabled test, "send and receive more ten times the size of a buffer" (`unix-domain-socket/src/test/scala/docs/scaladsl/UnixDomainSocketSpec.scala:87`). Enabling it shows the connector stalls part way through a large transfer: the stream neither completes nor fails, and the test times out after a minute.

The test has never run. It was added already marked `ignore` in b0b16175c ("Various tidy-ups", 2019-02-28), back in the Alpakka days, so this is not a regression — the transfer path it covers has simply never been exercised in CI.

## Reproduction

Change `ignore` to `in` and run:

```
sbt "unix-domain-socket/testOnly docs.scaladsl.UnixDomainSocketSpec -- -z \"ten times\""
```

The test echoes a single 640 KiB `ByteString` (`64 * 1024 * 10`) through `bindAndHandle(Flow.fromFunction(identity), path, halfClose = true)` and folds the response back together.

Result:

```
[info] - should send and receive more ten times the size of a buffer *** FAILED *** (1 minute)
[info] A timeout occurred waiting for a future to complete. Waited 1 minute. (UnixDomainSocketSpec.scala:104)
```

Line 104 is `result.futureValue`, i.e. the echoed-bytes future never completes.

## What the debug log shows

With `loglevel = DEBUG` the io thread logs every socket read. Across both ends of the connection there are exactly 33 reads, all of 8192 bytes (~270 KB in total, against 640 KB expected each way), and then it goes completely silent for the remaining minute:

```
16:35:00.595 DEBUG [default-dispatcher-5] Socket(pekko://UnixDomainSocketSpec) read: 8192
16:35:00.596 DEBUG [default-dispatcher-5] Socket(pekko://UnixDomainSocketSpec) read: 8192
16:35:00.597 DEBUG [default-dispatcher-8] Socket(pekko://UnixDomainSocketSpec) read: 8192
16:35:00.598 DEBUG [default-dispatcher-6] Socket(pekko://UnixDomainSocketSpec) read: 8192

```

No exception, no "Read-side is shutting down" or "Write-side is shutting down" message — everything simply stops. That points at a stall in the flow control rather than an error path: both directions of the connection are driven by the one selector in the extension, and an echo puts the client in the position of writing while it is also the only thing draining the response.

Config in play is the default `pekko.stream.connectors.unix-domain-socket.receive-buffer-size = 64k` / `send-buffer-size = 64k`.

## Why it matters

Sustained transfer is untested. The smaller tests in this spec push a handful of bytes and would not notice a flow-control problem. I recently added "not deliver received bytes before the consumer asks for them, and resume reading afterwards" to the same spec, which does exercise the read-side backpressure handshake, but deliberately stays well short of filling a socket buffer so that it stays fast — it does not cover whatever this test hits.

Ideally this test gets fixed and re-enabled rather than left disabled for another six years.

Contributor guide

Open the contributing guide

Research direction

Start with unix-domain-socket/src/test/scala/docs/scaladsl/UnixDomainSocketSpec.scala:87 and run the provided sbt test command after changing `ignore` to `in`. Use the DEBUG read logs and the connector's flow-control and selector behavior to identify why the echo stalls. Done means the 640 KiB transfer completes reliably and the throughput test is re-enabled in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.