Revisit benchmarker for HTTP benchmarks
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 399
- Forks
- 11
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
While analyzing the performance improvement of Node.js 20, I faced an unexpected behavior, see:
$ ./node18 benchmark/run.js --filter "simple.js" http
http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="test-double-http": 57,011
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="test-double-http": 54,078
$ ./node20 benchmark/run.js --filter "simple.js" http
http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="test-double-http": 127,857
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="test-double-http": 116,360
A noticeable performance improvement (from 57,011 to 127,857), but if you switch the benchmarker method, this performance drops:
$ npm i -g autocannon
added 48 packages in 5s
5 packages are looking for funding
run `npm fund` for details
$ ./node18 benchmark/run.js --filter "simple.js" http
http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="autocannon": 42,648.73
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="autocannon": 43,250.91
$ ./node20 benchmark/run.js --filter "simple.js" http
http/simple.js
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="bytes" benchmarker="autocannon": 46,552.73
http/simple.js duration=10 chunkedEnc=0 c=100 chunks=4 len=1024 type="buffer" benchmarker="autocannon": 46,477.1
This indicates that the previous benchmarking method (test-double-http) had limitations or errors. It might be due to the keep-alive turned on by default after Node.js 19?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with benchmark/run.js and the http/simple.js benchmark, then reproduce the Node.js 18 and 20 results using both test-double-http and autocannon. Investigate whether differing keep-alive behavior explains the discrepancy; done means the benchmarker's limitation or error is identified and the benchmark results or method are corrected accordingly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100