google / google/jsonnet

std.manifestJson is slow and memory inefficient, should be implemented as a native builtin

Open
#828 4 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Jsonnet
Stars
7.6k
Forks
475
PR merge metrics
No merged PRs in 30d

Description

I have a use case where I need to serialize multiple large objects to JSON. I'm currently using `std.manifestJson` to do this. I'm finding this both slow, and memory inefficient.

Here's a representative benchmark

Short-circuiting manifestation using `std.manifestJson({})`:
```
% /usr/bin/time -l do-my-work # this time implementation is built into macOS
5.16 real 13.77 user 3.63 sys
166146048 maximum resident set size # peak memory footprint in bytes
0 average shared memory size
0 average unshared data size
0 average unshared stack size
400646 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
41 messages sent
113 messages received
0 signals received
134 voluntary context switches
16460 involuntary context switches
```

Using `std.manifestJson()`:
```
% /usr/bin/time -l do-my-work
11.85 real 20.41 user 3.86 sys
247922688 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
435074 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
41 messages sent
105 messages received
0 signals received
222 voluntary context switches
30451 involuntary context switches
```

As you can see, this more than doubles the execution time and multiplies the memory footprint by around 1.5x.

It's hard to imagine that JSON manifestation would have this same impact if implemented natively in C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.