nodejs / nodejs/performance

buffer: optimize Buffer.toString('hex') for small buffers with stack allocation

Open
#194 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
399
Forks
11
Avg merge
29m
Merged PRs (30d)
1

Description

While looking into buffer optimizations (nodejs/node#61871), I noticed that Buffer.toString('hex') heap allocates a temporary buffer for every call even for tiny inputs like a SHA-256 digest

I tried using a 512 byte stack buffer for small inputs (<=256 bytes / 512 hex chars) and calling NewFromOneByte directly, skipping ExternOneByteString entirely. This covers all the common cases SHA-256, SHA-512, UUIDs, HMAC signatures, MongoDB ObjectIds, etc.

Let me know what you guys think of this (runs=60)

                                                                    confidence improvement accuracy (*)   (**)  (***)
buffers/buffer-tostring.js n=1000000 len=1 args=1 encoding='hex'           ***     34.44 %       ±1.05% ±1.39% ±1.80%
buffers/buffer-tostring.js n=1000000 len=1 args=3 encoding='hex'           ***     32.38 %       ±2.10% ±2.79% ±3.61%
buffers/buffer-tostring.js n=1000000 len=1024 args=1 encoding='hex'        ***     -1.11 %       ±0.62% ±0.82% ±1.07%
buffers/buffer-tostring.js n=1000000 len=1024 args=3 encoding='hex'                -0.84 %       ±0.88% ±1.17% ±1.51%
buffers/buffer-tostring.js n=1000000 len=128 args=1 encoding='hex'         ***     29.00 %       ±1.67% ±2.22% ±2.88%
buffers/buffer-tostring.js n=1000000 len=128 args=3 encoding='hex'         ***     30.72 %       ±0.79% ±1.05% ±1.35%
buffers/buffer-tostring.js n=1000000 len=256 args=1 encoding='hex'         ***     26.65 %       ±0.93% ±1.23% ±1.59%
buffers/buffer-tostring.js n=1000000 len=256 args=3 encoding='hex'         ***     27.13 %       ±1.46% ±1.93% ±2.49%
buffers/buffer-tostring.js n=1000000 len=512 args=1 encoding='hex'          **     -1.24 %       ±0.85% ±1.12% ±1.44%
buffers/buffer-tostring.js n=1000000 len=512 args=3 encoding='hex'          **     -0.63 %       ±0.41% ±0.55% ±0.71%
buffers/buffer-tostring.js n=1000000 len=64 args=1 encoding='hex'          ***     37.50 %       ±1.65% ±2.18% ±2.81%
buffers/buffer-tostring.js n=1000000 len=64 args=3 encoding='hex'          ***     39.28 %       ±1.29% ±1.72% ±2.23%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 12 comparisons, you can thus
expect the following amount of false-positive results:
  0.60 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.12 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the buffers/buffer-tostring.js benchmark and review the linked nodejs/node#61871 discussion for the proposed Buffer.toString('hex') optimization. Run the listed small- and large-buffer cases; done means validating the improvement for small inputs without introducing regressions for larger ones.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.