Shreemanarjun / Shreemanarjun/nitro_server

perf: reactor (kqueue/epoll) to scale past thread-per-connection

Open
#1 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Problem

nitro_server does not gain throughput from more isolates on /hello, while dart:io scales linearly:

server 1 isolate 4 isolates (shared)
dart:io ~32k req/s ~65-78k req/s
nitro ~58k req/s ~58-62k req/s

nitro leads at 1 isolate (~1.8×) but ties/trails once dart:io uses shared: true across 4 isolates.

Measured root cause

A standalone experiment (N independent ServerInstances on one port, C++ raw client, respond inlined on the worker — no Dart) shows the engine's ceiling on an M1 Pro (8 cores):

engines req/s
1 108k
2 108k
4 110k
8 110k

So one engine already saturates the box at ~108k and adding engines does not scale — SO_REUSEPORT-per-isolate is ruled out as a fix. The gap between 108k (engine) and 58k (real nitro) is the async Dart round-trip plus thread-per-connection: ~64 worker threads park in poll() on 8 cores. sample profiles are poll-dominated. More connections do not help (they add threads, raising latency).

Proposal

Replace thread-per-connection with an event-driven reactor: one kqueue/epoll thread per isolate watches all connection fds; a small worker pool (≈ core count) handles ready fds and returns them, instead of one parked thread per connection. This decouples connections from threads and should approach the ~108k engine ceiling, also lifting the "hundreds not thousands of connections" limit.

A Poller scaffold exists in commit 2df34f3. Large change; must preserve the direct-write answer path, WebSocket, streaming, sendfile, TLS, and drain semantics, all currently covered by the C++ and Dart suites.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Poller scaffold in commit 2df34f3 and inspect the existing thread-per-connection path and poll-dominated profiles. Run the existing C++ and Dart suites before changing the reactor. Done means the direct-write answer path, WebSocket, streaming, sendfile, TLS, and drain semantics remain covered while connection handling scales beyond the current thread limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, dart, flutter
Domain
backend, networking, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.