elastic / elastic/apm-agent-nodejs

Improve uuid performance

Open
#511 12 comments 4 reactions 0 assignees View on GitHub
agent-nodejs discuss performance
Dominant language
JavaScript
Stars
594
Forks
244
Avg merge
1d 8h
Merged PRs (30d)
16

Description

Each transaction is currently assigned a uuid v4. We use the [uuid](https://www.npmjs.com/package/uuid) module which internally uses `crypto.randomBytes(16)` to generate the uuid.

This is currently one of our biggest performance drains when creating new transactions, so I ran some benchmarks to see if there were any faster alternatives.

Here's the best(🔥) synchronous performers of the benchmark compared our current method(🎯):

| Method | Leaky | Format | Re-use | Cache | Ops/sec | RME |
|--------|-------|--------|--------|-------|---------|-----|
| [uuid/v1] | 💦 | uuid | | n/a | 1,074,518 | ±1.17% |
| [uuid/v1] | 💦 | hex | | n/a | 404,325 | ±1.63% |
| [uuid/v1] | 💦 | hex | ✅ | n/a | 545,523 | ±1.14% |
| [uuid/v4]🎯 | | uuid | | n/a | 233,282 | ±2.16% |
| [uuid/v4] | | hex | | n/a | 207,050 | ±4.65% |
| [uuid/v4] | | hex | ✅ | n/a | 243,580 | ±3.50% |
| [fast-uuid] | 💦 | uuid | | n/a | 630,719 | ±0.83% |
| [uuid-random]🔥 | | uuid | | 512 | 1,214,445 | ±0.42% |
| [uuid-random]🔥 | | hex | | 512 | 1,159,229 | ±0.80% |
| [sodium-uuid] | | hex | | n/a | 214,087 | ±0.51% |
| [sodium-uuid] | | hex | ✅ | n/a | 220,409 | ±0.54% |
| [hyperid]🔥 | 💦 | custom | | n/a | 8,843,573 | ±1.52% |
| [crypto.randomBytes] | | hex | | n/a | 243,894 | ±3.27% |
| [crypto.randomBytes] | | hex | | 256 | 932,165 | ±1.11% |
| [crypto.randomBytes] | | hex | ✅ | 256 | 897,471 | ±0.95% |
| [crypto.randomBytes]🔥 | | hex | | 4096 | 1,221,396 | ±0.64% |
| [crypto.randomBytes]🔥 | | hex | ✅ | 4096 | 1,140,619 | ±0.42% |
| [crypto.randomBytes]🔥 | | hex | | 65536 | 1,199,742 | ±4.04% |
| [crypto.randomBytes]🔥 | | hex | ✅ | 65536 | 1,163,009 | ±0.63% |
| [crypto.randomFillSync] | | hex | ✅ | n/a | 309,164 | ±0.52% |
| [crypto.randomFillSync] | | hex | | 256 | 1,053,831 | ±0.77% |
| [crypto.randomFillSync] | | hex | ✅ | 256 | 1,012,593 | ±0.52% |
| [crypto.randomFillSync]🔥 | | hex | | 4096 | 1,318,476 | ±0.69% |
| [crypto.randomFillSync]🔥 | | hex | ✅ | 4096 | 1,253,235 | ±0.39% |
| [crypto.randomFillSync]🔥 | | hex | | 65536 | 1,294,847 | ±1.67% |
| [crypto.randomFillSync]🔥 | | hex | ✅ | 65536 | 1,295,193 | ±1.05% |
| [/dev/random] | | hex | ✅ | n/a | 228,396 | ±0.34% |
| [/dev/random] | | hex | ✅ | 256 | 446,458 | ±0.62% |
| [/dev/random] | | hex | ✅ | 4096 | 482,517 | ±0.89% |
| [/dev/random] | | hex | ✅ | 65536 | 540,828 | ±0.75% |
| [/dev/urandom] | 💦 | hex | ✅ | n/a | 227,162 | ±0.72% |
| [/dev/urandom] | 💦 | hex | ✅ | 256 | 447,749 | ±0.46% |
| [/dev/urandom] | 💦 | hex | ✅ | 4096 | 490,723 | ±0.49% |
| [/dev/urandom] | 💦 | hex | ✅ | 65536 | 542,039 | ±0.87% |

_View all the results here and the source code for the benchmark here: https://github.com/watson/uuid-benchmark_

- **Method:** Name of npm module, Node.js core function, or OS based random generator used
- **Leaky:** Indicates that the method used to generate the 128 bit number doesn't leak metadata
- **Format:**
- `uuid` - ID is a UUID formatted string representing a 128 bit number, e.g. `3a017fc5-4f50-4db9-b0ce-4547ba0a1bfd`
- `hex` - ID is a hex formatted string representing a 128 bit number, e.g. `3a017fc54f504db9b0ce4547ba0a1bfd`
- `custom` - ID is **not** a UUID or 128 bit number, but is still globally unique
- **Re-use:** Indicates an output buffer was re-used between each test to potentially reduce the number of objects that needed to be created
- **Cache:**
- `n/a` - Only the amount of bytes required to generate a 128 bit number was read into memory
- <Number> - Number of bytes read into memory the first time a 128 bit number was requested. Subsequent runs would use the leftover bytes in the cache until it had been depleted, at which time another chunk of bytes would be read into memory
- **Ops/sec:** Number of UUID's generated per second
- **RME:** The relative margin of error (expressed as a percentage of the mean)

## Gotcha

The `crypto.randomFillSync` function was added in Node.js v7.10.0. For older versions `crypto.randomBytes` can be used instead.

~~I'm not 100% certain that [uuid-random] is actually secure. The judgement is purely based on what's written in its readme.~~ I read the source code, and it looks fine.

## Discussion topics

- Are we ok with leaking some types of metadata to gain speed? E.g. a sequential number
- Are there any privacy/safety issues in using a cache to speed up generating multiple random numbers?

[uuid/v1]: https://www.npmjs.com/package/uuid
[uuid/v4]: https://www.npmjs.com/package/uuid
[fast-uuid]: https://www.npmjs.com/package/fast-uuid
[uuid-random]: https://www.npmjs.com/package/uuid-random
[sodium-uuid]: https://www.npmjs.com/package/sodium-uuid
[hyperid]: https://www.npmjs.com/package/hyperid
[crypto.randomBytes]: https://nodejs.org/api/crypto.html#crypto_crypto_randombytes_size_callback
[crypto.randomFillSync]: https://nodejs.org/api/crypto.html#crypto_crypto_randomfillsync_buffer_offset_size
[crypto.randomFill]: https://nodejs.org/api/crypto.html#crypto_crypto_randomfill_buffer_offset_size_callback
[/dev/random]: https://en.wikipedia.org/wiki//dev/random
[/dev/urandom]: https://en.wikipedia.org/wiki//dev/random

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.