Shreemanarjun / Shreemanarjun/nitro_server
perf: reduce per-request FFI round-trip cost
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Problem
Per-request latency is dominated by the FFI round-trip: the engine posts the request head to Dart via Dart_PostCObject (async, event-loop scheduled), the handler runs, then respond crosses back. This ~async hop is why nitro's sequential /hello (~76 µs) trails dart:io (~67 µs) and why load latency (~977 µs @ 64 conns) sits above dart:io (~816 µs). dart:io avoids the boundary by handling requests in-isolate.
Already done
- Request headers packed into one string, unpacked lazily (
packedHeaders). - Query parsed on first access.
- Per-isolate head batching (
RawIncomingBatch+ flat-combiningCombiner): heads that arrive while the previous post is in flight cross as one message. - Sync handlers answered inline (no extra
Future). - Leaf-call FFI (
FastCalls, ~0.18 µs/call). - Metrics bucket resolved once at dispatch (removed a redundant
_entryForper request on the answer path).
Addressable next steps
- Head as one zero-copy blob; decode method/path/params on access (headers + query already lazy). Expected small.
- Pre-encoded response heads for
constheader maps (skip re-encoding per answer). - Investigate a lower-latency C++→Dart signal than one
Dart_PostCObjectper batch (this is anitrobridge concern; may not be addressable here).
These reduce the Dart-side critical path; the async port hop itself is inherent to the C++-engine + Dart-handler design and is the harder ceiling (see the reactor issue).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by benchmarking the sequential /hello and load cases described in the issue, then read the request path around RawIncomingBatch, Combiner, packedHeaders, FastCalls, and Dart_PostCObject. A contribution is complete when one listed optimization is implemented and the benchmarks show its effect without changing request or response behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100