Shreemanarjun / Shreemanarjun/nitro_server
perf: reactor (kqueue/epoll) to scale past thread-per-connection
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Problem
nitro_server does not gain throughput from more isolates on /hello, while dart:io scales linearly:
| server | 1 isolate | 4 isolates (shared) |
|---|---|---|
| dart:io | ~32k req/s | ~65-78k req/s |
| nitro | ~58k req/s | ~58-62k req/s |
nitro leads at 1 isolate (~1.8×) but ties/trails once dart:io uses shared: true across 4 isolates.
Measured root cause
A standalone experiment (N independent ServerInstances on one port, C++ raw client, respond inlined on the worker — no Dart) shows the engine's ceiling on an M1 Pro (8 cores):
| engines | req/s |
|---|---|
| 1 | 108k |
| 2 | 108k |
| 4 | 110k |
| 8 | 110k |
So one engine already saturates the box at ~108k and adding engines does not scale — SO_REUSEPORT-per-isolate is ruled out as a fix. The gap between 108k (engine) and 58k (real nitro) is the async Dart round-trip plus thread-per-connection: ~64 worker threads park in poll() on 8 cores. sample profiles are poll-dominated. More connections do not help (they add threads, raising latency).
Proposal
Replace thread-per-connection with an event-driven reactor: one kqueue/epoll thread per isolate watches all connection fds; a small worker pool (≈ core count) handles ready fds and returns them, instead of one parked thread per connection. This decouples connections from threads and should approach the ~108k engine ceiling, also lifting the "hundreds not thousands of connections" limit.
A Poller scaffold exists in commit 2df34f3. Large change; must preserve the direct-write answer path, WebSocket, streaming, sendfile, TLS, and drain semantics, all currently covered by the C++ and Dart suites.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Poller scaffold in commit 2df34f3 and inspect the existing thread-per-connection path and poll-dominated profiles. Run the existing C++ and Dart suites before changing the reactor. Done means the direct-write answer path, WebSocket, streaming, sendfile, TLS, and drain semantics remain covered while connection handling scales beyond the current thread limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, dart, flutter
- Domain
- backend, networking, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100