nodejs / nodejs/performance

Revisit benchmarker for HTTP benchmarks

Open
#80 0 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
399
Forks
11
Avg merge
29m
Merged PRs (30d)
1

Description

While analyzing the performance improvement of Node.js 20, I faced an unexpected behavior, see:

$ ./node18 benchmark/run.js --filter "simple.js" http

http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="test-double-http": 57,011
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="test-double-http": 54,078

$ ./node20 benchmark/run.js --filter "simple.js" http

http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="test-double-http": 127,857
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="test-double-http": 116,360

A noticeable performance improvement (from 57,011 to 127,857), but if you switch the benchmarker method, this performance drops:

$ npm i -g autocannon

added 48 packages in 5s

5 packages are looking for funding
  run `npm fund` for details
$ ./node18 benchmark/run.js --filter "simple.js" http

http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="autocannon": 42,648.73
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="autocannon": 43,250.91
$ ./node20 benchmark/run.js --filter "simple.js" http

http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="autocannon": 46,552.73
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="autocannon": 46,477.1

This indicates that the previous benchmarking method (test-double-http) had limitations or errors. It might be due to the keep-alive turned on by default after Node.js 19?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with benchmark/run.js and the http/simple.js benchmark, then reproduce the Node.js 18 and 20 results using both test-double-http and autocannon. Investigate whether differing keep-alive behavior explains the discrepancy; done means the benchmarker's limitation or error is identified and the benchmark results or method are corrected accordingly.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.