forwardemail / forwardemail/supertest

[fix] Supertest attempts to bind a port that is already in use on a Mac, causing random failures

Open
#894 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
14.4k
Forks
782
PR merge metrics
No merged PRs in 30d

Description

## Describe the bug

**Node.js version:** v22.22.0

**OS version:** macOS (Darwin 25.5.0). Not reproducible on Linux — verified in a `node:22.22` container, see below.

**Description:** On macOS, random port assignment can pick a port that is already in use by another process. The requests are then forwarded to the actual process owning the port, resulting in persistent random test flakes and failures

## Actual behavior

Run the same large test suite (~1,400 jest tests, ~200 supertest requests per run) on MacOS 100 times. Approximately 12% of tests on my machine fail randomly with unexpected 404 or timeout errors. No errors are observed on Linux.

The root cause is that `Test.serverAddress` starts the test server with `app.listen(0)`, which binds the wildcard address (`[::]`), while the client URL is hardcoded to `http://127.0.0.1:`

On macOS, a wildcard bind succeeds even when another process already holds a `127.0.0.1`-specific listener on the same port** (Node sets `SO_REUSEADDR` on all server sockets, and BSD's conflict rules only compare exact address+port pairs — see nodejs/node#26274, closed as OS semantics). When that happens, the kernel routes the test's IPv4 connection to the most specific listener — the foreign process, not the supertest server.

Because the failing port depends only on the kernel's ephemeral-port counter at bind time, any unrelated change (new middleware, test reordering, dependency upgrades) reshuffles which tests fail — it looks exactly like a race condition in the app under test, which is what we chased for a long time.

Verified by instrumenting every HTTP server in the test workers with an identifying response header: the flaked responses came back without the header, while the test's own server was live and never received the request. `lsof` matched every traceable failing port to an IDE process listening on `127.0.0.1:`:

```
$ lsof -nP -iTCP -sTCP:LISTEN | grep -E '(idea|Code)'
idea 742 ... TCP 127.0.0.1:49490 (LISTEN) <- port of a flaked 404
idea 742 ... TCP 127.0.0.1:49712 (LISTEN) <- port of a flaked 404
Code 54231 ... TCP 127.0.0.1:60931 (LISTEN) <- port of two timed-out tests
Code 55585 ... TCP 127.0.0.1:58535 (LISTEN) <- port of a flaked 404
```

On my machine I traced test failures to requests going to IDE helper processes — IntelliJ's built-in server and VSCode helpers hold a dozen or so long-lived `127.0.0.1` listeners in the ephemeral port range. IntelliJ answers any path with HTTP 404; one VSCode helper accepts the connection and never responds (test-timeout).

Note that #667 reports a similar problem from the client side — spurious connection failures traced to `serverAddress`'s implicit ephemeral ports and `SO_REUSEADDR`. This issue is plausibly the root cause of such reports: the wildcard bind is what makes the port "available" while something else owns it on loopback.

## Expected behavior

Tests should pass consistently across multiple runs on MacOS

## Code to reproduce

To reproduce, execute the following gist on a Mac machine with a couple of open IntelliJ/VSCode projects
https://gist.github.com/myrosia/3a26e30c0fab4f808aaf5166fcf20e0d

Failing test as a gist (test + a preload demonstrating the suggested fix): https://gist.github.com/myrosia/661d0c52dc405a3c38816711111beef9
```js
use strict';

/**
* Failing test for: supertest's serverAddress() binds the wildcard address,
* so on macOS another process's 127.0.0.1 listener on the same ephemeral
* port silently answers the test's requests.
*
* A `before` hook starts a SEPARATE PROCESS that binds 127.0.0.1-specific
* "foreign" listeners (answering 404) on the ephemeral ports the kernel is
* about to hand out — the role IDE helper processes play on real developer
* machines. The test then makes plain supertest requests against an
* always-200 canary app and asserts every response came from that app.
*
* Expected: passes everywhere once supertest binds its ephemeral servers to
* 127.0.0.1 explicitly.
* Actual: fails on macOS — requests are answered by the foreign process.
* Passes on Linux, where the kernel refuses to hand a wildcard
* bind a port that is already held on loopback.
*
* Run: npm i supertest && node --test failing-test-shadowed-port.test.js
* (Uses node's built-in test runner; the describe/it/before/after shape maps
* directly onto mocha for inclusion in supertest's own suite.)
*/

const { describe, it, before, after } = require('node:test');
const assert = require('node:assert');
const net = require('node:net');
const { spawn } = require('node:child_process');
const request = require('supertest');

const FOREIGN_COUNT = 100;
const REQUESTS = 20;
const CANARY = `supertest-canary-${process.pid}`;

const app = (req, res) => {
res.statusCode = 200;
res.end(CANARY);
};

// Child: bind 127.0.0.1-specific listeners on every second upcoming
// ephemeral port (client source ports consume the gaps), print the bound
// ports as JSON when ready, stay alive until killed.
const CHILD_SRC = `
const http = require('http');
const base = Number(process.argv[1]);
let pending = ${FOREIGN_COUNT};
const bound = [];
for (let i = 0; i < ${FOREIGN_COUNT}; i++) {
const done = () => { if (--pending === 0) console.log(JSON.stringify(bound)); };
const srv = http.createServer((req, res) => {
res.statusCode = 404;
res.end('foreign-server');
});
srv.once('error', done); // port taken by a real process — skip it
srv.listen(base + 1 + i * 2, '127.0.0.1', () => { bound.push(srv.address().port); done(); });
}
setInterval(() => {}, 1 << 30);
`;

describe('supertest ephemeral test server', () => {
let child;

before(async () => {
// Find where the kernel's ephemeral port counter currently is.
const probe = net.createServer().listen(0);
await new Promise(resolve => probe.once('listening', resolve));
const base = probe.address().port;
await new Promise(resolve => probe.close(resolve));

// Start the foreign process and wait until its listeners are up.
child = spawn(process.execPath, ['-e', CHILD_SRC, String(base)], {
stdio: ['ignore', 'pipe', 'inherit'],
});
await new Promise(resolve => child.stdout.once('data', resolve));
});

after(() => {
if (child) child.kill();
});

it('every request is served by the server supertest started', async () => {
for (let i = 1; i <= REQUESTS; i++) {
const test = request(app)
.get('/ping')
.timeout({ response: 2500, deadline: 4000 });
const port = Number(new URL(test.url).port);

let res;
try {
res = await test;
} catch (err) {
// superagent may deliver foreign non-2xx responses as errors
assert.ok(
err.response,
`request ${i} to 127.0.0.1:${port} failed without a response: ${err.message}`
);
res = err.response;
}

assert.strictEqual(
`${res.status} ${res.text}`,
`200 ${CANARY}`,
`request ${i} to 127.0.0.1:${port} was answered by a foreign process, ` +
`not the server supertest started (got HTTP ${res.status} ${JSON.stringify(res.text)})`
);
}
});
});
```

## Suggested fix

Bind the ephemeral server to loopback explicitly — `listen(0, '127.0.0.1')` — so the kernel only assigns ports that are actually free on `127.0.0.1`, and the test server is the most-specific listener for the address the client connects to.

The complication: passing a host makes the bind asynchronous (it goes through `dns.lookup`), and `serverAddress` reads `server.address().port` synchronously on the next line. Possible fix (demonstrated in loopback-fix-preload.cjs` in the gist above) is to keep the bind synchronous with a pre-bound handle, the way Node's `cluster` module does: `server.listen(net._createServerHandle('127.0.0.1', 0, 4))`. Preloading it makes the failing test pass.

## Workaround

- Pass an **already-listening** server to `request()` instead of the app, bound to loopback: supertest then uses its address as-is and never calls `listen(0)`:

```js
const server = http.createServer(app);
beforeAll(done => server.listen(0, '127.0.0.1', done));
afterAll(done => server.close(done));
// ...
await request(server).get('/api/thing');
```

## Checklist

- [x] I have searched through GitHub issues for similar issues.
- [x] I have completely read through the README and documentation.
- [x] I have tested my code with the latest version of Node.js and this package and confirmed it is still not working.

Contributor guide

Open the contributing guide

Research direction

Start at the serverAddress entry point that calls app.listen(0), then compare it with the loopback-fix-preload.cjs approach from the reproduction gist. Run the supplied shadowed-port failing test repeatedly on macOS; done means ephemeral requests consistently reach supertest's own server without foreign-process responses.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.