rivet-dev / rivet-dev/actors

perf(runner): O(N) array lookup in requestToActor causes severe CPU bottleneck under high load

Open Beginner friendly
#5,581 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
6.1k
Forks
250
Avg merge
2d 8h
Merged PRs (30d)
96

Description

The Bug

In engine/sdks/typescript/runner/src/tunnel.ts, the Tunnel class uses an array to map incoming requests to their respective actors:

	#requestToActor: Array<{
		gatewayId: GatewayId;
		requestId: RequestId;
		actorId: string;
	}> = [];

Every single time a message or request chunk is received, the code does an O(N) linear scan over this array using .find() and arraysEqual():

	getRequestActor(
		gatewayId: GatewayId,
		requestId: RequestId,
	): RunnerActor | undefined {
		const entry = this.#requestToActor.find(
			(entry) =>
				arraysEqual(entry.gatewayId, gatewayId) &&
				arraysEqual(entry.requestId, requestId),
		);
// ...

Additionally, cleaning up the request performs an O(N) .findIndex() followed by an O(N) .splice().

The Impact

If there are 10,000 active concurrent requests/WebSockets on a single runner, every incoming event (e.g. HTTP body chunk, WS message) will iterate through 10,000 array elements doing byte-by-byte comparisons. This results in O(N^2) processing overhead and will easily peg the Node.js event loop at 100% CPU under load, severely degrading throughput.

The Fix

We should refactor #requestToActor to use a Map<string, string> (using a composite key like ${idToStr(gatewayId)}:${idToStr(requestId)}), converting all lookups, inserts, and deletes to O(1).


Hey @jog1t, I noticed this bottleneck while studying the TypeScript SDK runner architecture! Could you please assign this issue to me? I'd love to submit a PR to refactor this into an O(1) Map!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in engine/sdks/typescript/runner/src/tunnel.ts and inspect every use of Tunnel.#requestToActor, including getRequestActor, the .findIndex() cleanup, and insertion paths. Replace the array-based lookups and cleanup with the proposed composite-key Map using idToStr, then verify request, message-chunk, and cleanup behavior remains correct with constant-time access.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.