anthropics / anthropics/sandbox-runtime
Bug: the mux front-end TCP port is shared across Node `cluster` workers, violating srt's own per-process proxy model → intermittent 407 in restricted mode
- Ngôn ngữ chính
- TypeScript
- Star
- 5.2k
- Fork
- 439
- Merge trung bình
- 2 ngày 9 phút
- Pull request đã merge (30 ngày)
- 13
Mô tả
## Summary
srt is built around a **one-process-one-proxy** model: the mux backend binds a **pid-scoped** unix socket, and each process mints its **own** random `proxyAuthToken` (the code comment even says the token is set *"only when this process owns the proxy"*). But the mux **front-end** is bound with a plain `server.listen(port, '127.0.0.1')` — no `exclusive` flag. Under Node's `cluster` module (how PM2 `exec_mode: cluster` runs an app), `net.Server.listen()` is intercepted and the listen handle is **shared across all workers**, and the primary round-robins connections across them.
So a sandboxed command spawned by **worker A** (its env carries **A's** token) connects to the shared mux port, gets round-robined to **worker B**, and B's proxy checks A's token against **B's** `proxyAuthToken` → mismatch → 407:
```
HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: Basic realm="srt"
```
This is not a missing feature — it is an **internal inconsistency**: the front-end port sharing contradicts the per-process isolation the rest of the module (pid-scoped backend socket, per-process token) already assumes. The failure rate scales as ≈ `(N-1)/N` for `N` workers (e.g. ~67% at `N = 3` in the reproduction below); a retry often lands on the right worker and succeeds. Single-process (fork / 1-worker) deployments are unaffected.
## Root cause
**srt's design intent is per-process isolation.** Two places make this explicit:
1. The mux **backend** binds a **pid-scoped** unix socket — `sandbox/mux-proxy.js`:
```js
function unixSocketPath() {
return join(tmpdir(), `srt-mux-${process.pid}-${(backendSeq++).toString(36)}.sock`);
}
```
2. The **auth token is per-process**, and the comment frames it as process-owned — `sandbox/sandbox-manager.js`:
```js
// The auth token is only set when this process owns the proxy; an
// external proxy (config.network.httpProxyPort) handles its own auth ...
proxyAuthToken = config.network.httpProxyPort !== undefined
? undefined
: randomBytes(16).toString('hex'); // distinct per worker
```
**But the front-end binding breaks that assumption under `cluster`.** In `startMuxProxyServer()` — `sandbox/sandbox-manager.js`:
```js
await listenInRange(mux.server, p => mux.server.listen(p, '127.0.0.1'), portRange, ...);
// non-Windows: portRange is undefined -> listens on 127.0.0.1:0, no `exclusive` flag
```
`net.Server.listen()` under Node `cluster` defaults to `exclusive: false`, so the primary **shares one listen handle across all workers**. Every worker therefore reports the **same** `muxPort`, which each worker then bakes into the token URL for the commands *it* spawns — `sandbox/sandbox-utils.js`:
```js
const auth = proxyAuthToken ? `srt:${proxyAuthToken}@` : '';
// -> http_proxy=http://srt:@localhost:
```
Now worker A's command connects to the shared port, is round-robined to worker B, and B's auth check rejects A's token — `sandbox/http-proxy.js`:
```js
const checkAuth = (got) => {
if (!options.proxyAuthToken) return { ok: true };
const m = /^basic\s+([a-z0-9+/=]+)\s*$/i.exec(got ?? '');
if (!m) return { ok: false };
const decoded = Buffer.from(m[1], 'base64').toString('utf8');
const sep = decoded.indexOf(':');
if (sep <= 0 || decoded.slice(sep + 1) !== options.proxyAuthToken) { // A's token !== B's token
return { ok: false };
}
return { ok: true, encodedCommand: encodedCommandFromProxyUser(decoded.slice(0, sep)) };
};
```
→ 407, emitted at both the CONNECT and plain-HTTP paths with `realm="srt"`.
The backend is isolated per pid and the token is minted per process, but the **front-end port is the one piece that leaks across processes** — so the design's own invariant (a command always reaches the proxy that issued its token) is violated whenever srt runs under `cluster`.
## Minimal reproduction (pure `@anthropic-ai/sandbox-runtime`, no wrapper)
```js
// repro.mjs — run: node repro.mjs (in a project that has @anthropic-ai/sandbox-runtime installed)
import cluster from "node:cluster";
import { spawn } from "node:child_process";
import os from "node:os";
import { SandboxManager } from "@anthropic-ai/sandbox-runtime";
const N = 3, HOST = "example.com";
if (cluster.isPrimary) {
let d = 0; for (let i = 0; i < N; i++) cluster.fork();
cluster.on("exit", () => { if (++d === N) process.exit(0); });
} else {
await SandboxManager.initialize({
network: { allowedDomains: [HOST], deniedDomains: [] },
filesystem: { allowWrite: [os.tmpdir()], denyWrite: [], denyRead: [] },
}, undefined, false);
const wrun = (cmd) => new Promise(async (res) => {
const { argv, env } = await SandboxManager.wrapWithSandboxArgv(cmd, undefined, undefined, undefined);
const p = spawn(argv[0], argv.slice(1), { shell: false, env });
let o = ""; p.stdout.on("data", d => o += d); p.stderr.on("data", d => o += d); p.on("close", () => res(o));
});
const disc = await wrun("printenv http_proxy");
const port = (disc.match(/@[^:]+:(\d+)/) || [])[1];
const tok = (disc.match(/:([0-9a-f]{8})[0-9a-f]*@/) || [])[1];
const cls = (o) => /407|realm="srt"/i.test(o) ? "407" : /< HTTP\/1\.1 200/.test(o) ? "OK" : "OTHER";
const c = {};
for (let r = 0; r < 3; r++) {
const rs = await Promise.all(Array.from({ length: 8 }, () => wrun(`curl -sv --max-time 6 http://${HOST}/ 2>&1`)));
for (const o of rs) c[cls(o)] = (c[cls(o)] ?? 0) + 1;
}
console.error(`worker pid=${process.pid} muxPort=${port} token8=${tok} => ${JSON.stringify(c)}`);
process.exit(0);
}
```
Observed on **0.0.67** (3 workers):
```
worker pid=47253 muxPort=59010 token8=cef7c280 => {"407":16,"OK":8}
worker pid=47251 muxPort=59010 token8=11a649f1 => {"407":13,"OK":11}
worker pid=47252 muxPort=59010 token8=c8ed79fd => {"407":15,"OK":9}
```
All workers share **muxPort 59010**; each has a **distinct token**; ~61% (44/72) get 407. Running the same processes standalone (not via `cluster`) gives distinct ports and **0 × 407**.
## Still present on 0.0.71 (current `latest`)
The same `repro.mjs`, run against a clean install of **0.0.71**, reproduces identically:
```
worker pid=8502 muxPort=53775 token8=f22dc2b0 => {"407":17,"OK":5,"OTHER":2}
worker pid=8501 muxPort=53775 token8=f8ef7089 => {"407":14,"OK":7,"OTHER":3}
worker pid=8503 muxPort=53775 token8=409a04e9 => {"407":17,"OK":5,"OTHER":2}
```
All three workers again share one `muxPort` (53775) with distinct tokens; ~67% (48/72) get 407.
The relevant code is byte-for-byte unchanged from 0.0.67 to 0.0.71: the `mux.server.listen(p, '127.0.0.1')` bind (no `exclusive`), the pid-scoped backend socket, the per-worker `randomBytes(16)` token, and the `checkAuth` token comparison. The only proxy-path change across 0.0.68–0.0.71 is that `checkAuth` now returns `{ ok, encodedCommand }` (encoding the command into the proxy *user* field for attribution) — orthogonal to this bug; the password field is still strictly compared against the per-worker token, so cross-worker delivery still 407s. (Note 0.0.71 also added the same username/password auth to the SOCKS front-end, so a SOCKS deployment under `cluster` will exhibit the same cross-worker 407.)
## Confirming the shared handle is the trigger
A minimal `net` server under `cluster` shows the handle-sharing directly, and that `exclusive: true` fixes it:
```
listen({ port: 0, host: "127.0.0.1", exclusive: false }) -> worker ports [58956,58956,58956,58956] (distinct=1, shared)
listen({ port: 0, host: "127.0.0.1", exclusive: true }) -> worker ports [58957,58958,58959,58960] (distinct=4, private)
```
## Suggested fix
Both options **restore the per-process isolation the rest of the module already assumes**, and neither changes the exposure surface — the front-end stays bound to `127.0.0.1` either way; `exclusive` only controls whether sibling `cluster` workers share the socket, not who can reach it (the token remains the access control).
**Option 1 (minimal, verified):** bind the mux front-end with `exclusive: true` so each worker gets its own handle/port even under `cluster`:
```js
// sandbox-manager.js startMuxProxyServer()
await listenInRange(
mux.server,
p => mux.server.listen({ port: p, host: '127.0.0.1', exclusive: true }),
portRange, ...
);
```
The `exclusive: true` check above confirms this yields per-worker distinct ports under `cluster`. Each worker then injects its own `muxPort` into its own commands → no cross-worker delivery → no 407. Keeps the current TCP architecture.
**Option 2 (matches the existing design most closely):** bind the mux front-end on a **pid-scoped unix socket**, exactly as the mux *backend* already does (`srt-mux-${process.pid}-….sock`). Node `cluster` does not share pipe servers bound to distinct paths, so workers stay isolated by construction — and the front-end would then follow the same per-pid convention as the backend.
## Impact & workaround
Any consumer running srt in allowlist/restricted mode under Node `cluster` (PM2 `exec_mode: cluster` is a very common Node deployment) gets a high, intermittent 407 rate on **all** sandboxed egress — effectively unusable in cluster + restricted mode. Current workaround is to run one process per listener (PM2 `fork` mode behind a front LB, or one process per container/replica), which is a non-trivial topology change for cluster-based fleets. Upgrading through the latest release (0.0.71) does **not** address it.
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.