hashicorp / hashicorp/vault-benchmark
Panic decoding JSON reports with `review` command
- Dominant language
- Go
- Stars
- 107
- Forks
- 28
- PR merge metrics
- No merged PRs in 30d
Description
I'm attempting to review a benchmark report that I earlier saved as a JSON file, but `vault-benchmark` panics when deserializing the file:
```
$ vault-benchmark -version
vault-benchmark v0.3.0
$ vault-benchmark review -results_file benchmark-10nvs.json
error reading report: json: cannot unmarshal string into Go value of type vegeta.Metrics
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x104e83754]
goroutine 1 [running]:
github.com/hashicorp/vault-benchmark/benchmarktests.(*Reporter).ReportTerse(0x0, {0x10534e580, 0x140001a2008})
github.com/hashicorp/vault-benchmark/benchmarktests/reports.go:117 +0x64
github.com/hashicorp/vault-benchmark/command.(*ReviewCommand).Run(0x14000300540, {0x140001a6060, 0x2, 0x2})
github.com/hashicorp/vault-benchmark/command/review.go:109 +0x270
github.com/mitchellh/cli.(*CLI).Run(0x140000e0140)
github.com/mitchellh/cli@v1.1.5/cli.go:262 +0x4a8
github.com/hashicorp/vault-benchmark/command.RunCustom({0x140001a6050?, 0x10468dd1c?, 0x1058e86f8?}, 0x0?)
github.com/hashicorp/vault-benchmark/command/main.go:138 +0x730
github.com/hashicorp/vault-benchmark/command.Run(...)
github.com/hashicorp/vault-benchmark/command/main.go:41
main.main()
github.com/hashicorp/vault-benchmark/main.go:20 +0x54
```
The JSON report was created through something like this:
```
vault-benchmark run -config benchmark.hcl -report_mode json > benchmark.json
```
Contributor guide
Research direction
Start in command/review.go around ReviewCommand.Run and benchmarktests/reports.go around Reporter.ReportTerse, using the reported stack trace as the entry point. Reproduce the failure with the shown run and review commands, then trace the JSON decoding error into the nil reporter path. Done means reviewing a generated JSON report no longer panics and its decoding failure is handled clearly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100