Shreemanarjun / Shreemanarjun/nitro_server

perf: reduce per-request FFI round-trip cost

Open
#2 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Problem

Per-request latency is dominated by the FFI round-trip: the engine posts the request head to Dart via Dart_PostCObject (async, event-loop scheduled), the handler runs, then respond crosses back. This ~async hop is why nitro's sequential /hello (~76 µs) trails dart:io (~67 µs) and why load latency (~977 µs @ 64 conns) sits above dart:io (~816 µs). dart:io avoids the boundary by handling requests in-isolate.

Already done

  • Request headers packed into one string, unpacked lazily (packedHeaders).
  • Query parsed on first access.
  • Per-isolate head batching (RawIncomingBatch + flat-combining Combiner): heads that arrive while the previous post is in flight cross as one message.
  • Sync handlers answered inline (no extra Future).
  • Leaf-call FFI (FastCalls, ~0.18 µs/call).
  • Metrics bucket resolved once at dispatch (removed a redundant _entryFor per request on the answer path).

Addressable next steps

  1. Head as one zero-copy blob; decode method/path/params on access (headers + query already lazy). Expected small.
  2. Pre-encoded response heads for const header maps (skip re-encoding per answer).
  3. Investigate a lower-latency C++→Dart signal than one Dart_PostCObject per batch (this is a nitro bridge concern; may not be addressable here).

These reduce the Dart-side critical path; the async port hop itself is inherent to the C++-engine + Dart-handler design and is the harder ceiling (see the reactor issue).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by benchmarking the sequential /hello and load cases described in the issue, then read the request path around RawIncomingBatch, Combiner, packedHeaders, FastCalls, and Dart_PostCObject. A contribution is complete when one listed optimization is implemented and the benchmarks show its effect without changing request or response behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.