perf(runner): O(N) array lookup in requestToActor causes severe CPU bottleneck under high load
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 6.1k
- Forks
- 250
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 96
Description
The Bug
In engine/sdks/typescript/runner/src/tunnel.ts, the Tunnel class uses an array to map incoming requests to their respective actors:
#requestToActor: Array<{
gatewayId: GatewayId;
requestId: RequestId;
actorId: string;
}> = [];
Every single time a message or request chunk is received, the code does an O(N) linear scan over this array using .find() and arraysEqual():
getRequestActor(
gatewayId: GatewayId,
requestId: RequestId,
): RunnerActor | undefined {
const entry = this.#requestToActor.find(
(entry) =>
arraysEqual(entry.gatewayId, gatewayId) &&
arraysEqual(entry.requestId, requestId),
);
// ...
Additionally, cleaning up the request performs an O(N) .findIndex() followed by an O(N) .splice().
The Impact
If there are 10,000 active concurrent requests/WebSockets on a single runner, every incoming event (e.g. HTTP body chunk, WS message) will iterate through 10,000 array elements doing byte-by-byte comparisons. This results in O(N^2) processing overhead and will easily peg the Node.js event loop at 100% CPU under load, severely degrading throughput.
The Fix
We should refactor #requestToActor to use a Map<string, string> (using a composite key like ${idToStr(gatewayId)}:${idToStr(requestId)}), converting all lookups, inserts, and deletes to O(1).
Hey @jog1t, I noticed this bottleneck while studying the TypeScript SDK runner architecture! Could you please assign this issue to me? I'd love to submit a PR to refactor this into an O(1) Map!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in engine/sdks/typescript/runner/src/tunnel.ts and inspect every use of Tunnel.#requestToActor, including getRequestActor, the .findIndex() cleanup, and insertion paths. Replace the array-based lookups and cleanup with the proposed composite-key Map using idToStr, then verify request, message-chunk, and cleanup behavior remains correct with constant-time access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100