Performance regression between v0.20.0 and master
- Dominant language
- Go
- Stars
- 1.8k
- Forks
- 263
- PR merge metrics
- No merged PRs in 30d
Description
I ran into this while debugging an other mystery. Master at the time of writing is 923f51b. I tested this on Apple M1 hardware.
It appears that one of the recent changes has had a considerable performance impact, it is reproducible with the cpp benchmarks.
```
$ for i in v0.20.0 master; do git checkout $i; go build -o jsonnet-$i ./cmd/jsonnet; done
$ hyperfine --warmup 3 --parameter-list binary './jsonnet-v0.20.0,./jsonnet-master' '{binary} cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000'
Benchmark #1: ./jsonnet-v0.20.0 cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000
Time (mean ± σ): 201.7 ms ± 3.8 ms [User: 169.0 ms, System: 46.8 ms]
Range (min … max): 196.3 ms … 208.5 ms 13 runs
Benchmark #2: ./jsonnet-master cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000
Time (mean ± σ): 290.9 ms ± 2.3 ms [User: 239.1 ms, System: 71.4 ms]
Range (min … max): 287.6 ms … 294.1 ms 10 runs
Summary
'./jsonnet-v0.20.0 cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000' ran
1.44 ± 0.03 times faster than './jsonnet-master cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000'
```
Profiling:
```
$ for i in v0.20.0 master; do JSONNET_CPU_PROFILE=cpu-$i.pprof ./jsonnet-$i cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000; go tool pprof -svg -output cpu-$i.pprof.svg cpu-$i.pprof; done
```
These are short-lived runs, so I'm not sure how accurate the profiles are.
- [cpu-v0.20.0.pprof.svg](https://github.com/user-attachments/assets/7cd94f39-186f-438a-8e08-12b555867aff)
- [cpu-master.pprof.svg](https://github.com/user-attachments/assets/8c04ad99-0a13-43c8-85d7-ad0b39b9d711)
I also noticed during a larger build that jsonnet-master was using 3.5 cores, whereas jsonnet-v0.20.0 only used a single core.
Contributor guide
Assessment
This issue has not been assessed yet.